Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implement second/minute helpers for temporal #493

Merged
merged 3 commits into from
Jun 23, 2021
Merged

implement second/minute helpers for temporal #493

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
3554584
implement second/minute helpers for temporal
ovr Jun 23, 2021
ac9db7b
Update arrow/src/compute/kernels/temporal.rs
ovr Jun 23, 2021
c297f3f
Update arrow/src/compute/kernels/temporal.rs
ovr Jun 23, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 110 additions & 0 deletions arrow/src/compute/kernels/temporal.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,72 @@ where
Ok(b.finish())
}

/// Extracts the minutes of a given temporal array as an array of integers
pub fn minute<T>(array: &PrimitiveArray<T>) -> Result<Int32Array>
where
T: ArrowTemporalType + ArrowNumericType,
i64: std::convert::From<T::Native>,
{
let mut b = Int32Builder::new(array.len());
match array.data_type() {
&DataType::Date64 | &DataType::Timestamp(_, _) => {
for i in 0..array.len() {
if array.is_null(i) {
b.append_null()?;
} else {
match array.value_as_datetime(i) {
Some(dt) => b.append_value(dt.minute() as i32)?,
None => b.append_null()?,
}
}
}
}
dt => {
return {
Err(ArrowError::ComputeError(format!(
"minute does not support type {:?}",
dt
)))
}
}
}

Ok(b.finish())
}

/// Extracts the seconds of a given temporal array as an array of integers
pub fn second<T>(array: &PrimitiveArray<T>) -> Result<Int32Array>
where
T: ArrowTemporalType + ArrowNumericType,
i64: std::convert::From<T::Native>,
{
let mut b = Int32Builder::new(array.len());
match array.data_type() {
&DataType::Date64 | &DataType::Timestamp(_, _) => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure this is correct: when the timestamp is timezone-aware can we extract the second from its corresponding timezone-unaware equivalent?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 In general I think Rust/Arrow and DataFusion's support for timezone aware timestamps is pretty poor. I wonder if we could perhaps change this to DataType::Timestamp(_, None) and error if DataType::Timestamp(_, Some(tz))

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Filed #500 to track

for i in 0..array.len() {
if array.is_null(i) {
b.append_null()?;
} else {
match array.value_as_datetime(i) {
Some(dt) => b.append_value(dt.second() as i32)?,
None => b.append_null()?,
}
}
}
}
dt => {
return {
Err(ArrowError::ComputeError(format!(
"second does not support type {:?}",
dt
)))
}
}
}

Ok(b.finish())
}

#[cfg(test)]
mod tests {
use super::*;
Expand Down Expand Up @@ -184,4 +250,48 @@ mod tests {
assert!(!b.is_valid(1));
assert_eq!(2024, b.value(2));
}

#[test]
fn test_temporal_array_date64_minute() {
let a: PrimitiveArray<Date64Type> =
vec![Some(1514764800000), None, Some(1550636625000)].into();

let b = minute(&a).unwrap();
assert_eq!(0, b.value(0));
assert!(!b.is_valid(1));
assert_eq!(23, b.value(2));
}

#[test]
fn test_temporal_array_timestamp_micro_minute() {
let a: TimestampMicrosecondArray =
vec![Some(1612025847000000), None, Some(1722015847000000)].into();

let b = minute(&a).unwrap();
assert_eq!(57, b.value(0));
assert!(!b.is_valid(1));
assert_eq!(44, b.value(2));
}

#[test]
fn test_temporal_array_date64_second() {
let a: PrimitiveArray<Date64Type> =
vec![Some(1514764800000), None, Some(1550636625000)].into();

let b = second(&a).unwrap();
assert_eq!(0, b.value(0));
assert!(!b.is_valid(1));
assert_eq!(45, b.value(2));
}

#[test]
fn test_temporal_array_timestamp_micro_second() {
let a: TimestampMicrosecondArray =
vec![Some(1612025847000000), None, Some(1722015847000000)].into();

let b = second(&a).unwrap();
assert_eq!(27, b.value(0));
assert!(!b.is_valid(1));
assert_eq!(7, b.value(2));
}
}