Introduce Arrow.ToTimestamp for performant ZonedDateTime encoding

docs/src/manual.md

@@ -61,6 +61,10 @@ In the arrow data format, specific logical types are supported, a list of which
 
 Note that when `convert=false` is passed, data will be returned in Arrow.jl-defined types that exactly match the arrow definitions of those types; the authoritative source for how each type represents its data can be found in the arrow [`Schema.fbs`](https://github.com/apache/arrow/blob/master/format/Schema.fbs) file.
 
+One note on performance: when writing `TimeZones.ZonedDateTime` columns to the arrow format (via `Arrow.write`), it is preferrable to "wrap" the columns in `Arrow.ToTimestamp(col)`, as long
+as the column has `ZonedDateTime` elements that all share a common timezone. This ensures the writing process can know "upfront" which timezone will be encoded and is thus much more
+efficient and performant.
+
 #### Custom types
 
 To support writing your custom Julia struct, Arrow.jl utilizes the format's mechanism for "extension types" by allowing the storing of Julia type name and metadata in the field metadata. To "hook in" to this machinery, custom types can utilize the interface methods defined in the `Arrow.ArrowTypes` submodule. For example:

src/eltypes.jl

@@ -303,6 +303,26 @@ ArrowTypes.JuliaType(::Val{ZONEDDATETIME_SYMBOL}, S) = ZonedDateTime
 ArrowTypes.fromarrow(::Type{ZonedDateTime}, x::Timestamp) = convert(ZonedDateTime, x)
 ArrowTypes.default(::Type{TimeZones.ZonedDateTime}) = TimeZones.ZonedDateTime(1,1,1,1,1,1,TimeZones.tz"UTC")
 
+"""
+    Arrow.ToTimestamp(x::AbstractVector{ZonedDateTime})
+
+Wrapper array that provides a more efficient encoding of `ZonedDateTime` elements to the arrow format. In the arrow format,
+timestamp columns with timezone information are encoded as the arrow equivalent of a Julia type parameter, meaning an entire column
+_should_ have elements all with the same timezone. If a `ZonedDateTime` column is passed to `Arrow.write`, for correctness, it must
+scan each element to check each timezone. `Arrow.ToTimestamp` provides a "bypass" of this process by encoding the timezone of the
+first element of the `AbstractVector{ZonedDateTime}`, which in turn allows `Arrow.write` to avoid costly checking/conversion and
+can encode the `ZonedDateTime` as `Arrow.Timestamp` directly.
+"""
+struct ToTimestamp{A, TZ} <: AbstractVector{Timestamp{Meta.TimeUnit.MILLISECOND, TZ}}
+    data::A # AbstractVector{ZonedDateTime}
+end
+
+ToTimestamp(x::A) where {A <: AbstractVector{ZonedDateTime}} = ToTimestamp{A, Symbol(x[1].timezone)}(x)
+Base.IndexStyle(::Type{<:ToTimestamp}) = Base.IndexLinear()
+Base.size(x::ToTimestamp) = (length(x.data),)
+Base.eltype(::ToTimestamp{A, TZ}) where {A, TZ} = Timestamp{Meta.TimeUnit.MILLISECOND, TZ}
+Base.getindex(x::ToTimestamp{A, TZ}, i::Int) where {A, TZ} = convert(Timestamp{Meta.TimeUnit.MILLISECOND, TZ}, getindex(x.data, i))
+
 struct Interval{U, T} <: ArrowTimeType
     x::T
 end

test/runtests.jl

@@ -289,6 +289,12 @@ t = (
 tbl = Arrow.Table(Arrow.tobuffer(t))
 @test tbl.col1[1] == Dates.DateTime(1970)
 
+# 95; Arrow.ToTimestamp
+x = [ZonedDateTime(Dates.DateTime(2020), tz"Europe/Paris")]
+c = Arrow.ToTimestamp(x)
+@test eltype(c) == Arrow.Timestamp{Arrow.Flatbuf.TimeUnitModule.MILLISECOND, Symbol("Europe/Paris")}
+@test c[1] == Arrow.Timestamp{Arrow.Flatbuf.TimeUnitModule.MILLISECOND, Symbol("Europe/Paris")}(1577836800000)
+
 end # @testset "misc"
 
 end