You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
If a list of a nested struct is empty, the JSON reader should not create a null struct value.
To Reproduce
Try to write an array of below:
#[test]fnjson_list_roundtrip(){let json_content = r#" {"list": [{"ints": 1}]} {"list": [{}]} {"list": []} {"list": null} {"list": [{"ints": null}]} {"list": [null]} "#;let ints_struct =
DataType::Struct(vec![Field::new("ints", DataType::Int32, true)]);let list_type = DataType::List(Box::new(Field::new("item", ints_struct,true)));let list_field = Field::new("list", list_type,true);let schema = Arc::new(Schema::new(vec![list_field]));let builder = ReaderBuilder::new().with_schema(schema).with_batch_size(64);letmut reader = builder.build(std::io::Cursor::new(json_content)).unwrap();let batch = reader.next().unwrap().unwrap();let list_row = batch
.column(0).as_any().downcast_ref::<ListArray>().unwrap();let values = list_row.values();// the {"list": []} value gets interpreted as an empty struct, causing the length below to be 5assert_eq!(values.len(), 4);assert_eq!(values.null_count(), 1);}
Expected behavior
Empty lists should not create an empty/null struct as this can lead to incorrect writing of data as the list's child length would disagree with the offsets.
Describe the bug
If a list of a nested struct is empty, the JSON reader should not create a null struct value.
To Reproduce
Try to write an array of below:
Expected behavior
Empty lists should not create an empty/null struct as this can lead to incorrect writing of data as the list's child length would disagree with the offsets.
Additional context
This relates to #704 but doesn't yet fix it
The text was updated successfully, but these errors were encountered: