Adopt the delta kernel types

[rtyler]

What feels like long ago, @roeap imported some kernel types manually into the kernel module from the early work on delta-kernel-rs. Now delta_kernel has been released with a 0.0.1 we can move to adopting those symbols and begin incrementally adopting functionality from the kernel.

This issue is just about the types. I'm not expecting to pull any of the log replay or snapshot code in

[ion-elgreco]

I noticed the types were incorrect in kernel for timestamps. Would be good to backport fixes from delta-rs into kernel, otherwise we will have to fix some bugs again ^^

[rtyler]

@ion-elgreco we do indeed have some divergence 😱

How confident are you in our tests for timestamp support? 😄

[nicklan]

I noticed the types were incorrect in kernel for timestamps. Would be good to backport fixes from delta-rs into kernel, otherwise we will have to fix some bugs again ^^

We should definitely just fix those in kernel. @ion-elgreco , do you have links to the fixes by any chance? Or just the divergence so I can figure out what needs to change? I can probably figure it out but might be faster if you just point me at it.

Thanks!

[ion-elgreco]

@nicklan I fixed the decimal parsing to prevent decimals being parsed that are beyond 38 precision or scale, see here (https://github.com/delta-io/delta-rs/pull/2332/files):

delta-rs/crates/core/src/kernel/models/schema.rs

Lines 548 to 552 in 81593e9

	if precision > DECIMAL_MAX_PRECISION || scale > DECIMAL_MAX_SCALE {
	return Err(serde::de::Error::custom(format!(
	"Precision or scale is larger than 38: {}, {}",
	precision, scale
	)));

and here:

delta-rs/crates/core/src/kernel/models/schema.rs

Lines 628 to 635 in 81593e9

	pub fn decimal(precision: u8, scale: i8) -> Result<Self, ProtocolError> {
	if precision > DECIMAL_MAX_PRECISION || scale > DECIMAL_MAX_SCALE {
	return Err(ProtocolError::InvalidField(format!(
	"decimal({},{})",
	precision, scale
	)));
	}
	Ok(DataType::Primitive(PrimitiveType::Decimal(

Here for timestampNtz:

delta-rs/crates/core/src/kernel/expressions/eval.rs

Lines 49 to 51 in 81593e9

	Timestamp(val) => {
	Arc::new(TimestampMicrosecondArray::from_value(*val, num_rows).with_timezone("UTC"))
	}

and some parsing of multiple formats here: https://github.com/delta-io/delta-rs/pull/2383/files#diff-8e1a60f799cd1ad29923ee75871099a19c15ddfc00bef4617d82bdc02cbe3198

[roeap]

take

[roeap]

started work on this, we do require some updates on kernel first to be able to consume the types, but hoping that this will not be too invasive.