-
Notifications
You must be signed in to change notification settings - Fork 91
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
how to support convert json to avro value with specified schema? #154
Comments
I would recommend you use the The JSON you have might be complex and perhaps might not be straightforward to translate to avro. Or there could be an incompatibility with the schema. Or some feature missing in the library (unfortunately there are quite a few). Without a bit more info, it's a bit hard for me to help out more, I am afraid... |
I already use a c implement at work , and add enum type support which it missing. It can be switch to rust implementation easily.
https://github.com/grisha/json2avro/blob/master/json2avro.c
…---Original---
From: "Antonio Uccio Verardi"<notifications@github.com>
Date: Mon, Aug 10, 2020 00:34 AM
To: "flavray/avro-rs"<avro-rs@noreply.github.com>;
Cc: "larluo"<larluo@clojurians.org>;"Author"<author@noreply.github.com>;
Subject: Re: [flavray/avro-rs] how to support convert json to avro value with specified schema? (#154)
I would recommend you use the Writer and Reader if you can. to_avro_datum and from_avro_datum alone are not properly compliant to the avro spec, if used in the wrong way. Nevertheless, it might be OK for your use case, as I have never used that command line tool and I know nothing about it.
The JSON you have might be complex and perhaps might not be straightforward to translate to avro. Or there could be an incompatibility with the schema. Or some feature missing in the library (unfortunately there are quite a few). Without a bit more info, it's a bit hard for me to help out more, I am afraid...
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
This is avro-tools code: It basically comtains two-level format. |
I see, thanks for sharing more details. It seems a good use case for This is where the function you are calling is defined: Line 470 in d841c04
As you can see, this is pretty low level and it can fail if the avro schema is not valid as per the Avro spec. Another option you have is using https://github.com/flavray/avro-rs/blob/master/src/types.rs#L231 , which implements a very naive conversion from JSON value to Avro value. This also could fail if you need a more clever conversion because of your avro schema. One more option is https://serde.rs/transcode.html which could help you out with the transcoding. |
i want to implement the equal function for avro command line:
i can use [to_avro_datum] to write [Avro Value] to binary file.
but i can't find a way to convert [Json Value] to [Avro Value] with specified schema.
i can simple convert the [Json Value] to [Avro Value], but it's schema is not right.
maybe the [from_avro_datum] with[ json reader] can do it, but it's binding to bytes reader
The text was updated successfully, but these errors were encountered: