common: value encoding for list and struct is slower than key encoding

[Gun9niR]

In #5165, we have benchmarked the ser / de of key / value encoding. We discovered that ser/de of value encoding for List and Struct are slower than key encoding. The perf gap comes from the use of protobuf.

risingwave/src/common/src/util/value_encoding/mod.rs

Lines 74 to 104 in 3cd78e7

	fn serialize_value(value: ScalarRefImpl<'_>, mut buf: impl BufMut) {
	match value {
	ScalarRefImpl::Int16(v) => buf.put_i16_le(v),
	ScalarRefImpl::Int32(v) => buf.put_i32_le(v),
	ScalarRefImpl::Int64(v) => buf.put_i64_le(v),
	ScalarRefImpl::Float32(v) => buf.put_f32_le(v.into_inner()),
	ScalarRefImpl::Float64(v) => buf.put_f64_le(v.into_inner()),
	ScalarRefImpl::Utf8(v) => serialize_str(v.as_bytes(), buf),
	ScalarRefImpl::Bool(v) => buf.put_u8(v as u8),
	ScalarRefImpl::Decimal(v) => serialize_decimal(&v, buf),
	ScalarRefImpl::Interval(v) => serialize_interval(&v, buf),
	ScalarRefImpl::NaiveDate(v) => serialize_naivedate(v.0.num_days_from_ce(), buf),
	ScalarRefImpl::NaiveDateTime(v) => {
	serialize_naivedatetime(v.0.timestamp(), v.0.timestamp_subsec_nanos(), buf)
	}
	ScalarRefImpl::NaiveTime(v) => {
	serialize_naivetime(v.0.num_seconds_from_midnight(), v.0.nanosecond(), buf)
	}
	ScalarRefImpl::Struct(s) => {
	serialize_struct_or_list(s.to_protobuf_owned(), buf);
	}
	ScalarRefImpl::List(list) => {
	serialize_struct_or_list(list.to_protobuf_owned(), buf);
	}
	}
	}
	fn serialize_struct_or_list(bytes: Vec<u8>, mut buf: impl BufMut) {
	buf.put_u32_le(bytes.len() as u32);
	buf.put_slice(bytes.as_slice());
	}

Possible solutions:

• Just use key encoding instead
• Explore more efficient interfaces of protobuf
• Find some other data format

Related context:

• flamegraph
• bench result

[xiangjinwu]

Should be fixed by #4672

[waruto210]

The perf of ser/de of value encoding for List and Struct has improved a lot by #5880, but deserialization of value encoding for List is still slower than key encoding.
Should I close this issue?

[Gun9niR]

Should we investigate why de is still slower before closing this issue?

[waruto210]

Should we investigate why de is still slower before closing this issue?

According to #5897, this should be caused by lazy allocation.