"Searching for modified files..." phase is slow, CPU bound and single core #154
VorpalBlade
started this conversation in
Ideas
Replies: 2 comments 1 reply
-
|
Beta Was this translation helpful? Give feedback.
0 replies
-
The program which would need to be parallelized is Unless |
Beta Was this translation helpful? Give feedback.
1 reply
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
The phase "Searching for modified files..." is really slow (over a minute and a half for me on an AMD Ryzene 5600X with NVME storage) and it appears to be single core and CPU bound (not IO bound). This is quite a long and annoying cycle time.
It would be nice if it could be sped up using for example GNU Parallel (a bit like
xargs
but much improved). I had a look at the code, and I believe it should be possible to run with GNU Parallel using exported bash functions. The function would need to output to stdout how to update the global state (found_files
,orig_file_props
, etc). Since GNU Parallel (unlikexargs
) can be invoked to prevent parallel output clobbering each other this would be fine.I could make a PR for this, but I don't know your policy on adding new dependencies, and it would be a fair bit of work for something that might get rejected out of hand.
Beta Was this translation helpful? Give feedback.
All reactions