add support for intel crc32 instruction

[andrewcooke]

see https://crc32c.angeloflogic.com/performance/

[andrewcooke]

also the amd instruction (see same link)

[ysangkok]

some links:

• intel's paper on the PCLMULQDQ optimization
• a related paper on speeding up zlib

cloudflare patches:

• zlib PCLMULQDQ patch
• zlib crc32 instruction patch

intel patches:

• zlib PCLMULQDQ patch
• zlib crc32 instruction patch

and finally:

• linux pclmulqdq support

[andrewcooke]

thanks; i'm trying to find out how to call assembly from julia...

[andrewcooke]

http://accounts.unipg.it/~angelini/welcome_file/redalg.pdf

[andrewcooke]

https://groups.google.com/d/msg/julia-users/PffilD_uaeg/hkQh5U7KE2QJ

[andrewcooke]

http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20120528/143720.html

[stevengj]

I did some digging, and this seems to work:

crc32(crc::Int32, x::Int32) =
      Base.llvmcall(("declare i32 @llvm.x86.sse42.crc32.32.32(i32, i32)",
                     """%3 = call i32 @llvm.x86.sse42.crc32.32.32(i32 %0, i32 %1)                    
                        ret i32 %3"""), Int32, Tuple{Int32,Int32}, crc, x)
crc32(Int32(0), Int32(1))

(I haven't checked whether the result is correct, but it gives me an answer without complaining at least.)

[andrewcooke]

oh wow. thanks. tbh i don't think i ever got to the point where i understood how to use it... i will need to read the technical note again. but if you feel like posting a patch, that would be awesome!

[stevengj]

Will be gone for the next few days, but let me know if you can't figure it out. Note that there are related instructions that take inputs of different widths, IIRC.

[stevengj]

In Julia 0.6, you can just call Base.crc32c to get an efficient hardware-accelerated CRC32c; see JuliaLang/julia#18297

[stevengj]

Ping: now that 0.6 is almost out, you might want to call Base.crc32c for that checksum type in 0.6, since it was more than an order of magnitude faster than CRC.jl last time I checked.