HIVE-29383: Iceberg: [V3] Add support for timestamp with nanosecond precession

[ayushtkn]

What changes were proposed in this pull request?

Add support for timestamp with nanosecond precession

Why are the changes needed?

To support the new timestamp types introduced as part of iceberg v3

Does this PR introduce any user-facing change?

Yes, users can leverage timestamp_ns & timestamptz_ns data types for iceberg v3 tables

How was this patch tested?

UT

[Aggarwal-Raghav]

On your latest changes, select * is working now in beeline ✅, but casting to string is not working ❌, you can check by adding
SELECT CAST(ts_ns AS STRING) FROM t; in timestamp_ns.q

Caused by: java.lang.RuntimeException: Hive 2 Internal error: type = nanosecond timestamp
	at org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorConverter$TextConverter.convert(PrimitiveObjectInspectorConverter.java:520)
	at org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorConverter$TextConverter.convert(PrimitiveObjectInspectorConverter.java:412)
	at org.apache.hadoop.hive.ql.udf.generic.GenericUDFToString.evaluate(GenericUDFToString.java:57)
	at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:231)
	at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:80)
	at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:68)
	at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:87)

[ayushtkn]

fixed the cast query and added in the q file

[Aggarwal-Raghav]

For classes like IcebergNanosecondTimestampObjectInspectorHive3, IcebergNanosecondTimestampWithZoneObjectInspectorHive3 , we need to override methods like getPrimitiveJavaObject, getPrimitiveWritableObject etc, they should return specific subclass like TimestampNanoTZ or TimestampNano

Any thoughts on this comments? This is what I meant:

package org.apache.iceberg.mr.hive.serde.objectinspector;

import java.time.LocalDateTime;
import org.apache.hadoop.hive.common.type.TimestampNano;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;

public class TestIcebergNanosecondTimestampObjectInspectorHive3 {

  @Test
  public void testScenario() {
    IcebergNanosecondTimestampObjectInspectorHive3 oi =
        IcebergNanosecondTimestampObjectInspectorHive3.get();
    LocalDateTime now = LocalDateTime.now();

    Object tempObject = oi.getPrimitiveJavaObject(now);

    Assertions.assertInstanceOf(
        TimestampNano.class,
        tempObject,
        "getPrimitiveJavaObject should return an instance of TimestampNano");
  }

[ERROR]   TestIcebergNanosecondTimestampObjectInspectorHive3.testScenario:37 getPrimitiveJavaObject should return an instance of TimestampNano ==> Unexpected type, expected: <org.apache.hadoop.hive.common.type.TimestampNano> but was: <org.apache.hadoop.hive.common.type.Timestamp>

[Aggarwal-Raghav]

@ayushtkn , can you please check if these changes make sense addendum.patch

[sonarqubecloud]

Quality Gate passed

Issues
68 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[Aggarwal-Raghav]

@ayushtkn , i have found another issue with the join on nanosecond timestamp (specific to MapJoin). If you agree, can we create a new sub-JIRA to fix new issues?

[ayushtkn]

Let me know the issue & I can try fix it or we can create a followup once we merge this.

Should get some review as well, if folks are ok with the approach as well. We created a bunch of empty classes. Will try ask someone next week.

I need to test it myself as well post holidays. Was just trying as a POC. The idea was to keep the delta as low as possible and leverage TIMESTAMP mostly, I think have deviated bit from my initial plan.

[Aggarwal-Raghav]

create table t1 (id int, ts nanosecond timestamp) stored BY ICEBERG STORED AS PARQUET TBLPROPERTIES('format-version'='3');
create table t2 (id int, ts nanosecond timestamp) stored BY ICEBERG STORED AS PARQUET TBLPROPERTIES('format-version'='3');

insert into t1 values(1,'2025-12-30 11:19:00.123456789');
insert into t2 values(1,'2025-12-30 11:19:00.123456789');

SET hive.auto.convert.join=true;
select * from t1 join t2 where t1.ts=t2.ts;

Caused by: java.lang.NullPointerException: Cannot invoke "java.util.List.get(int)" because "structFieldObjectInspectors" is null
	at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:122)
	at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.<init>(StandardStructObjectInspector.java:111)
	at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:339)
	at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:322)
	at org.apache.hadoop.hive.ql.exec.JoinUtil.getRowContainer(JoinUtil.java:384)
	at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.initializeOp(CommonJoinOperator.java:326)
	at org.apache.hadoop.hive.ql.exec.AbstractMapJoinOperator.initializeOp(AbstractMapJoinOperator.java:78)
	at org.apache.hadoop.hive.ql.exec.MapJoinOperator.initializeOp(MapJoinOperator.java:174)
	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:360)
	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:549)
	at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:503)
	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:369)
	at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:332)