Skip to content

I/O: Read::bytes() is very slow, even on a BufReader #69

@Shnatsel

Description

@Shnatsel

The performance of Read::bytes() iterator is atrocious, even over a BufReader.

Here's a sample program calculating a sum of all bytes in the file with various approaches:

In this test Read::bytes() 10x slower than reading the whole file into memory up front and 20x slower than manually reading into a smallish buffer over and over. The performance gap is so large that even reading line-by-line with UTF-8 validation is still multiple times faster!

Benchmarked using hyperfine on this public domain book in plain text format repeated 50 times.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions