Pipe to head: Broken Pipe

[njnygaard]

Running cat works fine in the normal case getting a file from S3.
Same command with | head -n1 throws an error:

thread 'tokio-runtime-worker' panicked at 'failed printing to stdout: Broken pipe (os error 32)', library/std/src/io/stdio.rs:1016:9
stack backtrace:
   0: rust_begin_unwind
             at /rustc/a8314ef7d0ec7b75c336af2c9857bfaf43002bfc/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /rustc/a8314ef7d0ec7b75c336af2c9857bfaf43002bfc/library/core/src/panicking.rs:142:14
   2: std::io::stdio::print_to
             at /rustc/a8314ef7d0ec7b75c336af2c9857bfaf43002bfc/library/std/src/io/stdio.rs:1016:9
   3: std::io::stdio::_print
             at /rustc/a8314ef7d0ec7b75c336af2c9857bfaf43002bfc/library/std/src/io/stdio.rs:1028:5
   4: parquet2json::output_for_command
   5: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll
   6: tokio::runtime::task::core::CoreStage<T>::poll
   7: tokio::runtime::task::harness::Harness<T,S>::poll
   8: tokio::runtime::blocking::pool::Inner::run
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: JoinError::Panic(Id(13), ...)', /Users/njnygaard/.cargo/registry/src/github.com-1ecc6299db9ec823/parquet2json-2.0.1/src/main.rs:178:33
stack backtrace:
   0: rust_begin_unwind
             at /rustc/a8314ef7d0ec7b75c336af2c9857bfaf43002bfc/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /rustc/a8314ef7d0ec7b75c336af2c9857bfaf43002bfc/library/core/src/panicking.rs:142:14
   2: core::result::unwrap_failed
             at /rustc/a8314ef7d0ec7b75c336af2c9857bfaf43002bfc/library/core/src/result.rs:1785:5
   3: parquet2json::main::{{closure}}
   4: std::thread::local::LocalKey<T>::with
   5: tokio::park::thread::CachedParkThread::block_on
   6: tokio::runtime::thread_pool::ThreadPool::block_on
   7: tokio::runtime::Runtime::block_on
   8: parquet2json::main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Any ideas?

To be fair, this works well as a workaround:

parquet2json s3://[bucket]/[file].gz.parquet cat 2>/dev/null | head -n1

[jupiter]

Thanks for the report. I can see that others also encounter this situation (rust-lang/rust#46016) and I'm considering what to do as a fix. According to that post, it might be that we just want to suppress the error message in this case.

However, you can also achieve the same without piping to head, by using the the built-in limit -l, --limit <LIMIT> in parquet2json. E.g.

> parquet2json s3://[bucket]/[file].gz.parquet cat -l1

(You can find more documentation using parquet2json cat --help.)

[njnygaard]

RTFM. I should have dug a bit deeper in the help. Cheers for the great tool. I'll close this because there are two solutions and an upstream issue.