You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have searched in the issues and found no similar issues.
What would you like to be improved?
With a query in Trino, operations such as fetching metadata information require multiple loading tables. This can be affected by objective factors such as network latency, resulting in slower queries.
In the Iceberg format, this caching is already implemented (native Iceberg implementation). We have this cache for our Mix-format's keyed tables, and we should add this cache for our unkeyed tables as well. This cache is also applicable to queries on the base store and change store.
HuangFru
changed the title
[Improvement][Trino]: Add cache for loading table when get table meta in unkeyed table
[Improvement]: Add cache for loading table when get table meta in unkeyed table
Nov 13, 2023
The Mixed-Iceberg format has been refactored, and the refactored Mixed-Iceberg will directly use the Iceberg Catalog, making it possible to reuse the caching capability of the Iceberg Catalog. You can refer to this issue, #1336, to learn more about the refactored Mixed-Iceberg Catalog.
The core implementation of the refactored MixedIceberg Catalog will look like this: link to the code.
Native Iceberg's Caching Catalog is now only implemented on Engines(Spark, Flink), but we can reuse it. Looks like implementing a caching catalog on the refactored Mixed-Iceberg Catalog is no harm, but I think the trino module's cache also should be implemented because loading the mix table not only loads the iceberg table but also has some other operations.
Search before asking
What would you like to be improved?
With a query in Trino, operations such as fetching metadata information require multiple loading tables. This can be affected by objective factors such as network latency, resulting in slower queries.
In the Iceberg format, this caching is already implemented (native Iceberg implementation). We have this cache for our Mix-format's keyed tables, and we should add this cache for our unkeyed tables as well. This cache is also applicable to queries on the base store and change store.
How should we improve?
Add a cache for loading tables.
Are you willing to submit PR?
Subtasks
No response
Code of Conduct
The text was updated successfully, but these errors were encountered: