AvroError enum for arrow-avro crate #8759

nathaniel-d-ef · 2025-10-31T16:55:37Z

Which issue does this PR close?

Closes AvroError enum for arrow-avro crate #8746

Rationale for this change

The arrow-avro crate currently uses ArrowError throughout. This lacks the level of precision other crates in the project, such as Parquet, have.

What changes are included in this PR?

A new AvroError enum
Application of AvroError on all internal methods where ArrowError was previously used. Errors on pub methods at the API boundary remain as ArrowError. A Result utility on the AvroError allows for Result<T, AvroError> to be written as Result.

Are these changes tested?

No new functionality has been introduced, all existing tests are passing.

Are there any user-facing changes?

There shouldn't be - the API signatures remain the same.

…ghout arrow-avro crate.

alamb

Thanks @nathaniel-d-ef -- I think this makes sense to me, but it is a breaking API change and thus we will have to wait until the next major release in a few months:

https://github.com/apache/arrow-rs/milestone/6

FYI @jecsand838 and @mbrobbel

alamb · 2025-10-31T21:15:59Z

arrow-avro/src/errors.rs

+    EOF(String),
+    /// Arrow error.
+    /// Returned when reading into arrow or writing from arrow.
+    ArrowError(String),


Could this instead keep the actual Arrow error rather than converting directly into a string?

Suggested change

ArrowError(String),

ArrowError(Box<ArrowError>)),

I realize that is what ParquetError::ArrowError does the same thing -- but I think we might want to change that Parquet error as well

alamb · 2025-10-31T21:17:41Z

arrow-avro/src/errors.rs

+    }
+}
+
+impl From<cell::BorrowMutError> for AvroError {


this is a fairly specific conversion -- maybe it would be simpler to annotate the locations where this happens with map_err(|e| AvroError::From(Box::new(e)) 🤔

Whoops, my mistake. This is unnecessary, I removed.

alamb · 2025-10-31T21:19:08Z

arrow-avro/src/writer/encoder.rs

        }
-        return out
-            .write_all(&src_be[extra..])
-            .map_err(|e| ArrowError::IoError(format!("write decimal fixed: {e}"), e));


This change does appear to lose some error context. Would it be better to keep the information that this came from write decimal fixed?

Yes, that's a good point. I adjusted this to use the General error and pass the contextual info.

jecsand838 · 2025-10-31T22:48:04Z

@nathaniel-d-ef I'll do a deeper review tonight, but if we plan to go this direction, should we also remove the AvroError variant from https://github.com/apache/arrow-rs/blob/main/arrow-schema/src/error.rs?

I never got around to wiring that up before public release. But the original intent was to align arrow-avro with arrow-csv and arrow-json.

CC: @alamb

jecsand838 · 2025-10-31T22:56:35Z

arrow-avro/src/errors.rs

+impl From<ArrowError> for AvroError {
+    fn from(e: ArrowError) -> AvroError {
+        AvroError::External(Box::new(e))
+    }
+}


@nathaniel-d-ef I'd consider doing something like this:

Suggested change

impl From<ArrowError> for AvroError {

fn from(e: ArrowError) -> AvroError {

AvroError::External(Box::new(e))

}

}

pub enum AvroError {

// ...

ArrowError(Box<ArrowError>),

// ...

}

impl From<ArrowError> for AvroError {

fn from(e: ArrowError) -> Self {

AvroError::ArrowError(Box::new(e))

}

}

impl std::error::Error for AvroError {

fn source(&self) -> Option<&(dyn Error + 'static)> {

match self {

AvroError::External(e) => Some(e.as_ref()),

AvroError::ArrowError(e) => Some(e.as_ref()),

_ => None,

}

}

}

jecsand838 · 2025-10-31T22:57:19Z

arrow-avro/src/errors.rs

+impl From<AvroError> for ArrowError {
+    fn from(p: AvroError) -> Self {
+        Self::AvroError(format!("{p}"))
+    }
+}


Then down here, you can also do this:

Suggested change

impl From<AvroError> for ArrowError {

fn from(p: AvroError) -> Self {

Self::AvroError(format!("{p}"))

}

}

impl From<AvroError> for ArrowError {

fn from(e: AvroError) -> Self {

match e {

AvroError::External(inner) => ArrowError::from_external_error(inner),

AvroError::ArrowError(inner) => ArrowError::from_external_error(inner),

other => ArrowError::AvroError(other.to_string()),

}

}

}

nathaniel-d-ef · 2025-11-03T13:59:39Z

@nathaniel-d-ef I'll do a deeper review tonight, but if we plan to go this direction, should we also remove the AvroError variant from https://github.com/apache/arrow-rs/blob/main/arrow-schema/src/error.rs?

I never got around to wiring that up before public release. But the original intent was to align arrow-avro with arrow-csv and arrow-json.

CC: @alamb

I see what you mean. I don't feel strongly either way. On the whole we're probably unlikely to use all of the variants here, so if it's preferable to roll this back a bit and implement the ArrowError::AvroError instead, I'm happy to go that direction. I mainly didn't want it stuck somewhere in the middle. Your comment on the issue is a good one, perhaps Parquet is more of the odd one out.

nathaniel-d-ef added 2 commits October 30, 2025 20:37

adds AvroError variant to arrow-avro crate

e5c1b54

Replace ArrowError with AvroError and adjust error handling throu…

3ac8f87

…ghout arrow-avro crate.

github-actions bot added arrow Changes to the arrow crate arrow-avro arrow-avro crate labels Oct 31, 2025

alamb approved these changes Oct 31, 2025

View reviewed changes

alamb added api-change Changes to the arrow API next-major-release the PR has API changes and it waiting on the next major version labels Oct 31, 2025

jecsand838 reviewed Oct 31, 2025

View reviewed changes

mbrobbel added this to the 58.0.0 milestone Nov 3, 2025

nathaniel-d-ef added 2 commits November 3, 2025 17:21

Cleanup, apply suggestions

6fb7248

Remove redundant newline

b40bb9c

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

AvroError enum for arrow-avro crate #8759

AvroError enum for arrow-avro crate #8759

Uh oh!

nathaniel-d-ef commented Oct 31, 2025

Uh oh!

alamb left a comment

Uh oh!

alamb Oct 31, 2025

Uh oh!

alamb Oct 31, 2025

Uh oh!

nathaniel-d-ef Nov 3, 2025

Uh oh!

alamb Oct 31, 2025

Uh oh!

nathaniel-d-ef Nov 3, 2025

Uh oh!

jecsand838 commented Oct 31, 2025 •

edited

Loading

Uh oh!

jecsand838 Oct 31, 2025

Uh oh!

jecsand838 Oct 31, 2025

Uh oh!

nathaniel-d-ef commented Nov 3, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

-impl From<ArrowError> for AvroError {
-    fn from(e: ArrowError) -> AvroError {
-        AvroError::External(Box::new(e))
-    }
-}
+pub enum AvroError {
+    // ...
+    ArrowError(Box<ArrowError>),
+    // ...
+}
+impl From<ArrowError> for AvroError {
+    fn from(e: ArrowError) -> Self {
+        AvroError::ArrowError(Box::new(e))
+    }
+}
+impl std::error::Error for AvroError {
+    fn source(&self) -> Option<&(dyn Error + 'static)> {
+        match self {
+            AvroError::External(e) => Some(e.as_ref()),
+            AvroError::ArrowError(e) => Some(e.as_ref()),
+            _ => None,
+        }
+    }
+}

-impl From<AvroError> for ArrowError {
-    fn from(p: AvroError) -> Self {
-        Self::AvroError(format!("{p}"))
-    }
-}
+impl From<AvroError> for ArrowError {
+    fn from(e: AvroError) -> Self {
+        match e {
+            AvroError::External(inner) => ArrowError::from_external_error(inner),
+            AvroError::ArrowError(inner) => ArrowError::from_external_error(inner),
+            other => ArrowError::AvroError(other.to_string()),
+        }
+    }
+}

AvroError enum for arrow-avro crate #8759

Are you sure you want to change the base?

AvroError enum for arrow-avro crate #8759

Uh oh!

Conversation

nathaniel-d-ef commented Oct 31, 2025

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

alamb left a comment

Choose a reason for hiding this comment

Uh oh!

alamb Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

alamb Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

nathaniel-d-ef Nov 3, 2025

Choose a reason for hiding this comment

Uh oh!

alamb Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

nathaniel-d-ef Nov 3, 2025

Choose a reason for hiding this comment

Uh oh!

jecsand838 commented Oct 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jecsand838 Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

jecsand838 Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

nathaniel-d-ef commented Nov 3, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jecsand838 commented Oct 31, 2025 •

edited

Loading