Iceberg vs Hive performance

[yusufshalaby]

Loving the package. Thanks to all the maintainers. I'm curious how people are picking between the two table formats.

I was excited about implementing Iceberg initially, but I'm finding my dbt runs are very slow, especially on a partitioned table. Maybe the partition metadata and file management creates a performance bottleneck? Or maybe Athena/Trino is not optimized for Iceberg. I'm just speculating, but I'm wondering what other users' experiences have been. Will probably move my pipeline to Hive and compare performance in the future.

Also, dumb question, but how can I join the community slack?

[roslovets]

I tried to use the Iceberg format with this adapter, and it really worked (thanks to amazing contributors)!
But it was so sloooow as you have mentioned. So I made a step back to Hive and now I'm building only Hive tables.
So I'm also questioning why to use Iceberg in Athena. Is ACID really needed for dbt transformation?

@nicor88 , @svdimchenko , @Jrmyy Could you share your experience please? It would be really interesting to know if you are building some tables in Iceberg.