You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The du example swallows all errors when reading a directory, and it shows under massive parallelism as jwalk tends to run out of filedescriptors. Steps to reproduce:
edit the example to have parallelism 500 instead of 4
set ulimit -n 5 -- this step is sometimes not required, but may be required on systems with really large fd limit.
launch cargo run --example du on a large folder repeatedly and observe indeterministic bytecount without any printed errors
Here is a fix for that:
diff --git a/examples/du.rs b/examples/du.rs
index 6e51703..b62dbe8 100644
--- a/examples/du.rs+++ b/examples/du.rs@@ -23,6 +23,10 @@ fn main() {
{
match dir_entry_result {
Ok(dir_entry) => {
+ if let Some(ref err) = dir_entry.read_children_error {+ panic!("error: {}", err);+ }+
if let Some(len) = &dir_entry.client_state {
eprintln!("counting {:?}", dir_entry.path());
total += len;
This is a duplicate of #4 -- I don't think this is good API at all, there's a user in that issue confused by it and jwalk's own examples get its usage wrong.
Additionally, if I try to observe read_children_error in process_read_dir, it is always None regardless of whether jwalk ran into ulimits.
The text was updated successfully, but these errors were encountered:
The
du
example swallows all errors when reading a directory, and it shows under massive parallelism as jwalk tends to run out of filedescriptors. Steps to reproduce:ulimit -n 5
-- this step is sometimes not required, but may be required on systems with really large fd limit.cargo run --example du
on a large folder repeatedly and observe indeterministic bytecount without any printed errorsHere is a fix for that:
This is a duplicate of #4 -- I don't think this is good API at all, there's a user in that issue confused by it and jwalk's own examples get its usage wrong.
Additionally, if I try to observe
read_children_error
inprocess_read_dir
, it is alwaysNone
regardless of whether jwalk ran into ulimits.The text was updated successfully, but these errors were encountered: