Skip to content

Commit

Permalink
Buffer file reads in serde_json::from_reader (#10341)
Browse files Browse the repository at this point in the history
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

https://docs.rs/serde_json/latest/serde_json/fn.from_reader.html
suggests that

> When reading from a source against which short reads are not
efficient, such as a
[File](https://doc.rust-lang.org/std/fs/struct.File.html), you will want
to apply your own buffering because serde_json will not buffer the
input. See
[std::io::BufReader](https://doc.rust-lang.org/std/io/struct.BufReader.html).

Without this buffering, we observe a sequence of single byte reads which
can be quite inefficient depending on the underlying filesystem.

This adds buffering with `std::io::BufReader` to resolve this.


<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

Unit tests cover this code.

<!-- How was it tested? -->
  • Loading branch information
ajtulloch authored Jan 7, 2025
1 parent a1a4d82 commit 671e938
Showing 1 changed file with 5 additions and 2 deletions.
7 changes: 5 additions & 2 deletions crates/uv-distribution-types/src/installed.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
use std::borrow::Cow;
use std::io::BufReader;
use std::path::{Path, PathBuf};
use std::str::FromStr;

Expand Down Expand Up @@ -309,7 +310,8 @@ impl InstalledDist {
Err(err) if err.kind() == std::io::ErrorKind::NotFound => return Ok(None),
Err(err) => return Err(err.into()),
};
let direct_url = serde_json::from_reader::<fs_err::File, DirectUrl>(file)?;
let direct_url =
serde_json::from_reader::<BufReader<fs_err::File>, DirectUrl>(BufReader::new(file))?;
Ok(Some(direct_url))
}

Expand All @@ -321,7 +323,8 @@ impl InstalledDist {
Err(err) if err.kind() == std::io::ErrorKind::NotFound => return Ok(None),
Err(err) => return Err(err.into()),
};
let cache_info = serde_json::from_reader::<fs_err::File, CacheInfo>(file)?;
let cache_info =
serde_json::from_reader::<BufReader<fs_err::File>, CacheInfo>(BufReader::new(file))?;
Ok(Some(cache_info))
}

Expand Down

0 comments on commit 671e938

Please sign in to comment.