Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support reading json string into binary data type. #701

Closed
zijie0 opened this issue Aug 20, 2021 · 0 comments · Fixed by #702
Closed

Support reading json string into binary data type. #701

zijie0 opened this issue Aug 20, 2021 · 0 comments · Fixed by #702
Labels

Comments

@zijie0
Copy link
Contributor

zijie0 commented Aug 20, 2021

Describe the bug

When decoding json string into binary data type, arrow-rs will return JsonError("Binary type is not supported").

To Reproduce

Running the following test could reproduce the error.

#[test]
fn test_json_read_binary_structs() {
    let schema = Schema::new(vec![Field::new("c1", DataType::Binary, true)]);
    let decoder = Decoder::new(Arc::new(schema), 1024, None);
    let batch = decoder
        .next_batch(
            &mut vec![
                Ok(serde_json::json!({
                    "c1": "₁₂₃",
                })),
                Ok(serde_json::json!({
                    "c1": "foo",
                })),
            ]
            .into_iter(),
        )
        .unwrap()
        .unwrap();
    let data = batch.columns().iter().collect::<Vec<_>>();

    let schema = Schema::new(vec![Field::new("c1", DataType::Binary, true)]);
    let binary_values = BinaryArray::from(vec![
        "\u{2081}\u{2082}\u{2083}".as_bytes(),
        "foo".as_bytes(),
    ]);
    let expected_batch =
        RecordBatch::try_new(Arc::new(schema), vec![Arc::new(binary_values)])
            .unwrap();
    let expected_data = expected_batch.columns().iter().collect::<Vec<_>>();

    assert_eq!(data, expected_data);
    assert_eq!(batch.num_columns(), 1);
    assert_eq!(batch.num_rows(), 2);
}

Expected behavior

The above test should pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant