Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spark Doris Connector Release Note 24.0.0 #243

Open
JNSimba opened this issue Dec 27, 2024 · 0 comments
Open

Spark Doris Connector Release Note 24.0.0 #243

JNSimba opened this issue Dec 27, 2024 · 0 comments

Comments

@JNSimba
Copy link
Member

JNSimba commented Dec 27, 2024

Spark Connector

Feature

Support spark doris catalog. #239
Support more syntax pushdown. #219
Support group commit. #224
Support for reading via Arrow Flight SQL. #227
Supports reading bitmap types by converting to strings through Arrow Flight SQL. #239
Refactor the code structure to adapt to the features of different spark versions. #239

Improvement

Optimize memory usage during compressed transmission. #212
Optimize default read batch size. #230
Support custom Thrift maximum message size. #233
Supports custom stream load label prefix. #235

BugFix

Fixed the issue of missing check condition for stream load status exception. #206
Fixed the issue of spark date type conversion exception. #209
Fixed Bitmap and HLL type metadata issues. #218
Fix the problem of reading and deserializing time type. #221
Fix the incompatibility issue after deserialization of time type. #226
Fix the time zone issue when reading datetime type. #232

Spark Load

New Spark Load #214

Thanks

@gnehil
@JNSimba
@liutang123
@liugddx
@smallhibiscus
@vinlee19
@wary
@zhaorongsheng

@JNSimba JNSimba pinned this issue Dec 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant