[x86-64] Bad codegen for certain SIMD intrinsics

[dcommander]

I am in the process of converting the x86-64 SIMD modules in libjpeg-turbo to compiler intrinsics (libjpeg-turbo/libjpeg-turbo#732.) However, I am encountering a problem whereby Clang/LLVM tries to outsmart me when it translates certain intrinsics into assembly code, and the resulting assembly code is often not smart at all. Here is a good example:

https://godbolt.org/z/7YGrvGbnc

Clang translates the two intrinsics into seven assembly instructions, whereas GCC correctly translates them into the two assembly instructions that correspond to the intrinsics. The result is that, when compiled with GCC, the intrinsics version of our color conversion algorithm performs as well as the NASM version, but when compiled with Clang, the intrinsics version regresses by 20-30%.

With AVX2, Clang translates the equivalent two intrinsics into two assembly instructions, but they are slower instructions than the instructions that correspond to the intrinsics:

https://godbolt.org/z/nzx7f16e7

If someone goes to the trouble of writing intrinsics that have a documented 1:1 correspondence with assembly instructions, it's because they are trying to talk to the hardware more directly. The compiler really shouldn't second guess them in that case.

Is there a way to disable this behavior?

I have tried all of the -O options, to no avail. I did observe that changing -msse2 to -mssse3 (or targeting any later SIMD instruction set, such as AVX2) causes Clang to compile _mm_slli_si128() and _mm_unpackhi_epi8() into vpshufd and vpunpcklbw rather than pslldq and punpckhbw. That behavior is inscrutable, though, since vpshufd and vpunpcklbw are both SSE2 instructions.

[llvmbot]

@llvm/issue-subscribers-backend-x86

Author: DRC (dcommander)

I am in the process of converting the x86-64 SIMD modules in libjpeg-turbo to compiler intrinsics (libjpeg-turbo/libjpeg-turbo#732.) However, I am encountering a problem whereby Clang/LLVM tries to outsmart me when it translates certain intrinsics into assembly code, and the resulting assembly code is often not smart at all. Here is a good example:

https://godbolt.org/z/7YGrvGbnc

Clang translates the two intrinsics into seven assembly instructions, whereas GCC correctly translates them into the two assembly instructions that correspond to the intrinsics. The result is that, when compiled with GCC, the intrinsics version of our color conversion algorithm performs as well as the NASM version, but when compiled with Clang, the intrinsics version regresses by 20-30%.

With AVX2, Clang translates the equivalent two intrinsics into two assembly instructions, but they are slower instructions than the instructions that correspond to the intrinsics:

https://godbolt.org/z/nzx7f16e7

If someone goes to the trouble of writing intrinsics that have a documented 1:1 correspondence with assembly instructions, it's because they are trying to talk to the hardware more directly. The compiler really shouldn't second guess them in that case.

Is there a way to disable this behavior?

I have tried all of the -O options, to no avail. I did observe that changing -msse2 to -mssse3 (or targeting any later SIMD instruction set, such as AVX2) causes Clang to compile _mm_slli_si128() and _mm_unpackhi_epi8() into vpshufd and vpunpcklbw rather than pslldq and punpckhbw. That behavior is inscrutable, though, since vpshufd and vpunpcklbw are both SSE2 instructions.

[RKSimon]

Clang moved to implementing many intrinsics as generic IR instructions a long time ago, permitting a lot of aggressive optimisation - that's not going to change now.

I'll take a look at your particular codegen regressions

[fhahn]

To clarify, this isn't an actual mis-compile (usually meaning incorrect code), just worse code than expected?

[RKSimon]

just a perf regression - the shuffles are getting combined, either in vectorcombine or in a generic dag fold and then x86 shuffle lowering has to split them apart and doesn't match the original shuffles

[dcommander]

To clarify, this isn't an actual mis-compile (usually meaning incorrect code), just worse code than expected?

Intel's documentation strongly implies a 1:1 correspondence between SIMD intrinsics and assembly instructions. They even go to the trouble of listing instruction latency and throughput for each intrinsic, which cannot be true unless the intrinsic is compiled to its literal assembly counterpart. The code that LLVM generates produces mathematically correct results, but I would argue that it isn't correct because, in multiple cases, it fails to honor the documented correspondence between SIMD intrinsics and SIMD assembly instructions. If developers cannot rely on that correspondence, then SIMD intrinsics are not very useful for developing hand-tuned high-performance code, which is what I'm trying to do. Unfortunately that leaves me with few options:

1. Continue using NASM code.
2. Require GCC or Visual C++, which don't experience this issue (not really an option.)
3. Go through this process every time I encounter such an issue, wait for the fixes to be released in LLVM, and print a warning if users attempt to use an earlier version. (I have already spent 10-15 hours chasing down this issue, and that was just for one algorithm. The project would require overhauling 30 algorithms, and I have a limited budget for it.)

just a perf regression - the shuffles are getting combined, either in vectorcombine or in a generic dag fold and then x86 shuffle lowering has to split them apart and doesn't match the original shuffles

The original code interleaves the bytes from the lower halves of two SIMD registers (an important operation when converting RGB pixels to YCbCr.) It does so by left shifting the first register by 8 bytes and then unpacking the upper halves of the registers.

• With AVX2, LLVM inscrutably compiles the left shift and high unpack intrinsics to a blend and a shuffle. However, the overall performance of the algorithm is still faster than the original NASM version.
• With SSE2 and -msse2, LLVM inscrutably compiles the left shift and high unpack intrinsics to a series of seven instructions that expand the bytes to 16-bit words, perform the unpack, then re-pack the results to bytes.
• With SSE2 and -mssse3, LLVM compiles the left shift and high unpack intrinsics to a shuffle and a low unpack, which are at least performance-equivalent. However, it's unclear why it doesn't do this with -msse2, since the instructions generated here are SSE2 instructions.

One thing I tried was rewriting the algorithm to use shuffle and low unpack intrinsics, to simulate what LLVM does with -mssse3. However, LLVM still generated the aforementioned series of seven instructions for each set of shuffle and low unpack intrinsics.

The minimum necessary fix would be for LLVM to compile the shift and unpack intrinsics the same way with -msse2 as it does with -mssse3. However, per above, I am dubious that this will be the end of the required fixes. There are probably other translation issues that will only be discovered in the process of porting the other algorithms.

[topperc]

It's ugly, but you can trick LLVM by placing a dummy empty inline asm block between the intrinsics. https://godbolt.org/z/v54vcKj8e