Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
rust: write attachments in parts (#1285)
### Changelog <!-- Write a one-sentence summary of the user-impacting change (API, UI/UX, performance, etc) that could appear in a changelog. Write "None" if there is no user-facing change --> rust: support writing attachments in multiple chunks ### Docs <!-- Link to a Docs PR, tracking ticket in Linear, OR write "None" if no documentation changes are needed. --> None ### Description This change supports writing large attachments to MCAP files without the need to load them into memory. Currently, in order to write an attachment, you need to load the entire file into memory, create an `Attachment` record, then use that with the `attach` method on the writer. This means that it is not possible to attach files when the file exceeds the available memory on the computer. This change adds The methods `start_attachment`, `put_attachment_bytes` and `finish_attachment` for interactively writing an attachment without needing to buffer the file in memory. It looks like this: ```rust let writer = Writer::new(..); let attachment_length = ..; let header = AttachmentHeader { .. }; writer.start_attachment(attachment_length, header)?; writer.put_attachment_bytes(..)?; writer.put_attachment_bytes(..)?; writer.put_attachment_bytes(..)?; writer.finish_attachment()?; ``` I've updated the current `attach` method to use this approach and the roundtrip tests pass. I've also added some tests for using this API directly. <!-- Describe the problem, what has changed, and motivation behind those changes. Pretend you are advocating for this change and the reader is skeptical. --> <!-- In addition to unit tests, describe any manual testing you did to validate this change. --> <table><tr><th>Before</th><th>After</th></tr><tr><td> attachments must be loaded into memory to be written <!--before content goes here--> </td><td> attachments can be written in parts <!--after content goes here--> </td></tr></table> <!-- If necessary, link relevant Linear or Github issues. Use `Fixes: foxglove/repo#1234` to auto-close the Github issue or Fixes: FG-### for Linear isses. --> --------- Co-authored-by: james-rms <james@foxglove.dev>
- Loading branch information