[Improvement]: Use CachingCatalog to reduce the time cost of IcebergCatalogWrapper#loadTable

[zhongqishang]

Search before asking

• I have searched in the issues and found no similar issues.

What would you like to be improved?

Every time ArcticTable is obtained, it will go through a complete load table process(org.apache.iceberg.catalog.Catalog#loadTable). We can cache the loaded table to reduce the time cost of get ArcticTable.

How should we improve?

import org.apache.iceberg.CachingCatalog

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Subtasks

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[shidayang]

I am more concerned about how to obtain the latest state if there is caching.

[zhongqishang]

I am more concerned about how to obtain the latest state if there is caching.

Via refresh() to obtain the latest state?

[shidayang]

I think TableRuntimeRefreshExecutor must use the fresh table, Cache tables can be used as appropriate in other areas.
Does loadTable significantly impact performance now?

[zhoujinsong]

I believe that adding CacheCatalog can indeed reduce frequent access to the metadata system in some scenarios where tables need to be frequently loaded, such as frequent read operations on Dashboard.

However, if a refresh is performed every time a table is loaded, it seems that the overhead reduction may not be much compared to the cost of loading the table each time.

If this is true, we may need to determine where to force table refresh and manually refresh when needed.