is there a way to make writing timestamp_ntz optional

[djouallah]

currently Fabric don't support reading delta table with timestamp_ntz columns, is there a way to turn it off ?

[t1g0rz]

@djouallah

It seems to me that you can explicitly specify the timezone that you prefer. As far as I remember, in the previous version, there was a default UTC timezone. Now, you just need to add the timezone to the schema definition:

from deltalake import DeltaTable
import pyarrow as pa

DeltaTable.create('./test1', schema=pa.schema([pa.field('ts', pa.timestamp('us', tz='UTC'))]))
DeltaTable.create('./test2', schema=pa.schema([pa.field('ts', pa.timestamp('us'))]))

DeltaTable('./test1').schema()
# Schema([Field(ts, PrimitiveType("timestamp"), nullable=True)])

DeltaTable('./test2').schema()
# Schema([Field(ts, PrimitiveType("timestampNtz"), nullable=True)])

You can find a reference to pyarrow documentation here.

[ion-elgreco]

It's only enabled when you're passing timestamps with no timezone during write.

[djouallah]

Thanks, got me scared for a couple of hours, casting as TIMESTAMPTZ fixed the issue