Support csv truncated rows in datafusion

[zhuqi-lucas]

Which issue does this PR close?

Support csv truncated rows in datafusion

• Closes #17464

Rationale for this change

Support csv truncated rows in datafusion

What changes are included in this PR?

Support csv truncated rows in datafusion

Are these changes tested?

Yes

Are there any user-facing changes?

New feature support.

[xudong963]

LGTM

[alamb]

Thanks @zhuqi-lucas !

[zhuqi-lucas]

Thank you @xudong963 , @alamb!

[sm4rtm4art]

Hey,

I'm currently updating the DataFrame API documentation and noticed something about CsvReadOptions.

The method truncated_rows is publicly available in the source, but it doesn't seem to be rendered on the docs.rs page here:
https://docs.rs/datafusion/latest/datafusion/datasource/file_format/options/struct.CsvReadOptions.html

although it is implemented here :

https://github.com/apache/datafusion/blob/main/datafusion/core/src/datasource/file_format/options.rs#L94-L98

I just wanted to confirm if this is expected before I document it.

It seems like a useful builder method to mention.

Thanks!

[zhuqi-lucas]

Hey,

I'm currently updating the DataFrame API documentation and noticed something about CsvReadOptions.

The method with_truncated_rows is publicly available in the source, but it doesn't seem to be rendered on the docs.rs page here: https://docs.rs/datafusion/latest/datafusion/datasource/file_format/options/struct.CsvReadOptions.html

although it is implemented here :

https://github.com/apache/datafusion/blob/main/datafusion/core/src/datasource/file_format/options.rs#L94-L98

I just wanted to confirm if this is expected before I document it.

It seems like a useful builder method to mention.

Thanks!

It works well @sm4rtm4art , thank you!