Iceberg: Fail to get table stats when column lower/upper bound has NaN value

[lirui-apache]

Suppose we have an iceberg table populated with spark:

create table foo(d double) using iceberg;
insert into foo values (cast('NaN' as double));

Then executing show stats for foo with trino will throw exception:

Caused by: java.lang.IllegalArgumentException: min must not be NaN
	at io.trino.spi.statistics.DoubleRange.<init>(DoubleRange.java:61)
	at io.trino.spi.statistics.DoubleRange.from(DoubleRange.java:55)
	at io.trino.plugin.iceberg.TableStatisticsReader.makeTableStatistics(TableStatisticsReader.java:202)
	at io.trino.plugin.iceberg.TableStatisticsReader.getTableStatistics(TableStatisticsReader.java:101)
	at io.trino.plugin.iceberg.IcebergMetadata.lambda$getTableStatistics$65(IcebergMetadata.java:2350)
	at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1708)
	at io.trino.plugin.iceberg.IcebergMetadata.getTableStatistics(IcebergMetadata.java:2329)
	at io.trino.plugin.base.classloader.ClassLoaderSafeConnectorMetadata.getTableStatistics(ClassLoaderSafeConnectorMetadata.java:319)
	at io.trino.metadata.MetadataManager.getTableStatistics(MetadataManager.java:460)

[lirui-apache]

Maybe we should invalidate the stats if min/max is NaN?

[ebyhr]

I can't reproduce this issue on master. Could you share the Trino and Spark (Iceberg) versions?

[lirui-apache]

Hi @ebyhr , thanks for your comment. We're using iceberg 0.11.1. I checked newer versions and found a related PR [1]. And the spec also mentions NaN values are not allowed in column bounds [2]. So I guess we don't need to change on Trino side. Closing this issue.

[1] apache/iceberg#2464
[2] https://iceberg.apache.org/spec/#manifests