Add API to Retrieve Finished Writer from Parquet Writer #2498
Conversation
| @@ -223,6 +223,11 @@ impl<W: Write> ArrowWriter<W> { | |||
| Ok(()) | |||
| } | |||
|
|
|||
| /// Returns the underlying writer. | |||
| pub fn into_inner(self) -> W { | |||
There was a problem hiding this comment.
I think this should either call flush, or mention the need for this in the doc comment. What do you think?
There was a problem hiding this comment.
Thanks for reminding.
At first, I think caller should call flush before into_inner, but since this method consume ownership, so it' best we call flush inside it just like what close does.
parquet/src/file/writer.rs
Outdated
| @@ -306,6 +311,11 @@ impl<W: Write> SerializedFileWriter<W> { | |||
| Ok(()) | |||
| } | |||
| } | |||
|
|
|||
| /// Returns the underlying writer. | |||
| pub fn into_inner(self) -> W { | |||
There was a problem hiding this comment.
I think this needs to call self.write_metadata like close does, otherwise the file will not have a footer
tustvold
changed the title
|
I took the liberty of applying the doc changes, so that I can get this in for the next release |
|
Benchmark runs are scheduled for baseline = 15f42b2 and contender = f3afdd2. f3afdd2 is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
|
Thanks for quick response. |
Which issue does this PR close?
Closes #2491
Rationale for this change
What changes are included in this PR?
Add into_inner in ArrowWriter and its underlying Writer.
Are there any user-facing changes?
Yes, a new API
into_innerin ArrowWriter