Skip to content
This repository has been archived by the owner on Feb 18, 2024. It is now read-only.

Migrated from avro-rs to avro-schema #692

Merged
merged 1 commit into from
Dec 20, 2021
Merged

Migrated from avro-rs to avro-schema #692

merged 1 commit into from
Dec 20, 2021

Conversation

jorgecarleitao
Copy link
Owner

@jorgecarleitao jorgecarleitao commented Dec 19, 2021

This PR replaces avro-rs by avro-schema (authored by me).

Background

While working on reading from Avro some months ago, and now on #690, I noticed 4 issues with avro-rs, some of which are blocking:

  1. I was unable to create a Schema::Union since both its properties and the constructor are private :/
  2. I identified a bug which was causing avro-rs to write schemas incorrectly
  3. avro-rs drags dependencies and code to deserialize from Avro, but we do not use it since we implement deserializers directly to arrow
  4. avro-rs is unmaintained (its new home seems so as well), and some of my PRs there are not being merged or dragged upstream

This PR

Given the above, I wrote a new crate, avro-schema, containing the declaration of the Avro schema (as per spec) in Rust. This hopefully offers more flexibility and unblocks #690

The other added bonus are:

  1. The create only depends on serde and serde_json
  2. The crate is quite small (<800 LOC) and #![forbid(unsafe_code)]
  3. addresses all the issues mentioned above

@codecov
Copy link

codecov bot commented Dec 19, 2021

Codecov Report

Merging #692 (dc1784d) into main (f64339c) will decrease coverage by 0.04%.
The diff coverage is 46.51%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #692      +/-   ##
==========================================
- Coverage   69.77%   69.73%   -0.05%     
==========================================
  Files         303      303              
  Lines       16855    16865      +10     
==========================================
- Hits        11761    11760       -1     
- Misses       5094     5105      +11     
Impacted Files Coverage Δ
src/io/avro/mod.rs 0.00% <ø> (ø)
src/io/avro/read/util.rs 95.00% <ø> (ø)
src/io/avro/read/schema.rs 53.68% <40.00%> (+1.30%) ⬆️
src/io/avro/read/header.rs 84.61% <60.00%> (-15.39%) ⬇️
src/io/avro/read/deserialize.rs 66.66% <100.00%> (-5.99%) ⬇️
src/io/avro/read/mod.rs 81.25% <100.00%> (ø)
src/io/avro/read_async/metadata.rs 95.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f64339c...dc1784d. Read the comment docs.

@jorgecarleitao jorgecarleitao added enhancement An improvement to an existing feature and removed backwards-incompatible labels Dec 20, 2021
@jorgecarleitao jorgecarleitao merged commit 72c875c into main Dec 20, 2021
@jorgecarleitao jorgecarleitao deleted the avro_schema branch December 20, 2021 19:27
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement An improvement to an existing feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant