Panic in AsyncBufReadExt::fill_buf

[UebelAndre]

Version

tokio-panic v0.1.0 (/Users/andrebrisco/Code/tokio-panic)
└── tokio v1.10.1
    └── tokio-macros v1.3.0 (proc-macro)

Platform

Darwin Users-MacBook-Pro.local 20.6.0 Darwin Kernel Version 20.6.0: Wed Jun 23 00:26:31 PDT 2021; root:xnu-7195.141.2~5/RELEASE_X86_64 x86_64

Linux hostname.local 4.18.0-240.15.1.el8_3.x86_64 #1 SMP Mon Mar 1 17:16:16 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Description

When trying to implement an async way to generate a sha256 checksum for a file on disk, I run into a panic within the tokio library

Callstack

thread 'main' panicked at 'poll_fill_buf returned Pending while having data', /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.10.1/src/io/util/fill_buf.rs:44:34
stack backtrace:
   0: std::panicking::begin_panic
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/panicking.rs:541:12
   1: <tokio::io::util::fill_buf::FillBuf<R> as core::future::future::Future>::poll
             at /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.10.1/src/io/util/fill_buf.rs:44:34
   2: tokio_panic::calculate_sha256::{{closure}}
             at ./src/main.rs:15:26
   3: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/core/src/future/mod.rs:80:19
   4: tokio_panic::main::{{closure}}
             at ./src/main.rs:34:30
   5: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/core/src/future/mod.rs:80:19
   6: tokio::park::thread::CachedParkThread::block_on::{{closure}}
             at /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.10.1/src/park/thread.rs:263:54
   7: tokio::coop::with_budget::{{closure}}
             at /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.10.1/src/coop.rs:106:9
   8: std::thread::local::LocalKey<T>::try_with
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/thread/local.rs:400:16
   9: std::thread::local::LocalKey<T>::with
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/thread/local.rs:376:9
  10: tokio::coop::with_budget
             at /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.10.1/src/coop.rs:99:5
  11: tokio::coop::budget
             at /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.10.1/src/coop.rs:76:5
  12: tokio::park::thread::CachedParkThread::block_on
             at /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.10.1/src/park/thread.rs:263:31
  13: tokio::runtime::enter::Enter::block_on
             at /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.10.1/src/runtime/enter.rs:151:13
  14: tokio::runtime::thread_pool::ThreadPool::block_on
             at /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.10.1/src/runtime/thread_pool/mod.rs:71:9
  15: tokio::runtime::Runtime::block_on
             at /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.10.1/src/runtime/mod.rs:452:43
  16: tokio_panic::main
             at ./src/main.rs:34:5
  17: core::ops::function::FnOnce::call_once
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
The terminal process "cargo 'run', '--package', 'tokio-panic', '--bin', 'tokio-panic'" terminated with exit code: 101.

I'm able to reproduce this when running the code below on both my Mac and Linux machines.

Cargo.toml

[package]
name = "tokio-panic"
version = "0.1.0"
edition = "2018"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
rustc-serialize = "0.3.24"
sha2 = "0.9.6"
tokio = {version = "1.10.1", features = ["full"]}

src/main.rs

use rustc_serialize::hex::ToHex;
use sha2::{Digest, Sha256};
use std::path::{Path, PathBuf};
use tokio::fs::File;
use tokio::io::{AsyncBufReadExt, BufReader};

async fn calculate_sha256(path: &Path) -> String {
    let file = File::open(path).await.unwrap();

    let mut hasher = Sha256::new();
    let mut reader = BufReader::new(file);

    loop {
        let consumed = {
            let buffer = reader.fill_buf().await.unwrap();
            if buffer.is_empty() {
                break;
            }

            hasher.update(buffer);
            buffer.len()
        };

        reader.consume(consumed);
    }

    let checksum = hasher.finalize();
    checksum[..].to_hex()
}

#[tokio::main]
async fn main() {
    let path = PathBuf::from(concat!(env!("CARGO_MANIFEST_DIR"), "/src/data.txt"));
    println!("Checksum: {}", calculate_sha256(&path).await);
}

src/data.txt
Note that for src/data.txt, I generated this using touch src/data.txt (So it's otherwise an empty file)

Expectation
I'd obviously expect this not to panic but don't have high confidence I'm not doing something wrong.

[UebelAndre]

I believe this would be addressed by #4084