Skip to content

Commit

Permalink
lower default concurrency factor to 4
Browse files Browse the repository at this point in the history
I couldn't see a performance difference on newer kernels, faster on older ones, overall when not profiling I now do reliable 4GiB/s with peaks at 5GiB/s since moving from sha256.
The single core performance is where the most improvement is seen, when hashing really big files it used to do it at 800MiB/s it now does it at 1.6GiB/s.
  • Loading branch information
Jorropo committed May 3, 2024
1 parent d511e1e commit 3b7424c
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion main.go
Original file line number Diff line number Diff line change
Expand Up @@ -326,7 +326,7 @@ func dedup(backoff chan struct{}, length uint64, paths ...string) {
}

func main() {
concurrencyFactor := flag.Int("cf", 16, "define the concurrency factor, this allows to set the amount of workers run per linux core, use GOMAXPROCS env to configure the number of cores used.")
concurrencyFactor := flag.Int("cf", 4, "define the concurrency factor, this allows to set the amount of workers run per linux core, use GOMAXPROCS env to configure the number of cores used.")
cpuprofile := flag.String("cpuprofile", "", "write cpu profile to `file`")
onlyScan := flag.Bool("only-scan", false, "only scan and do not dedup")
flag.Parse()
Expand Down

0 comments on commit 3b7424c

Please sign in to comment.