-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add xxHash as a checksum option for checksum
and dedup
algorithms
#16503
Comments
Before discussing default change, the first step would to make it optional to measure its characteristics comparing to the others. And any algorithm added into the tree would have to stay there forever, so it must be really that good as advertised. |
Sorry I misread the man page. I thought |
checksum=on
default algorithm to xxHash (currently fletcher4)checksum
and dedup
algorithms
I think fletcher4 is a bit faster then current OpenZFS xxhash variants - so adding it as a new hash doesn't make sense. Here you have some fine table with hashes and their speeds: https://rurban.github.io/smhasher/doc/table.html
|
@mcmilk Your table indicates
With modern CPU so powerful, it makes sense to me to trade a bit of performance for much better sanity by replacing fletcher4 with xxh64. |
Why not sth. like Also, with
|
jfyi fletcher4 on
|
Describe the feature would like to see added to OpenZFS
Add xxHash as an option for
checksum
and bothxxhash
andxxhash,verify
fordedup
.How will this feature improve OpenZFS?
xxHash is sufficiently fast but much less collision-prone than fletcher4. This will improve ZFS resilience against silent data corruption as a competitive alternative to fletcher4.
Additional context
Performance as advertised by xxHash on its wiki: https://github.com/Cyan4973/xxHash/wiki/Performance-comparison (Note:
fletcher4
not included in this page)Collision ratio on xxHash wiki: https://github.com/Cyan4973/xxHash/wiki/Collision-ratio-comparison
The text was updated successfully, but these errors were encountered: