clang is slower than gcc when compiling some codes in chrome

[llvmbot]

Bugzilla Link	32037
Resolution	FIXED
Resolved on	Jan 24, 2018 03:57
Version	trunk
OS	Linux
Blocks	#24719
Attachments	row_common.cc, preprocessed
Reporter	LLVM Bugzilla Contributor
CC	@chandlerc,@topperc,@zmodem,@hfinkel,@lalozano,@RKSimon,@rnk,@rotateright
Fixed by commit(s)	323320

Extended Description

This is about compile time, not code quality.

When building chrome, there are some objects that take noticeably longer than gcc. Here are the top 5:

diff / clang-4.0 time / gcc-4.9.2 time / object
17.752848, 58.020705, 40.267857, obj/third_party/sqlite/chromium_sqlite3/sqlite3.o
10.927715, 15.727189, 4.799474, obj/third_party/libyuv/libyuv/row_common.o
8.964672, 19.499460, 10.534788, obj/v8/v8_base/wasm-interpreter.o
7.865181, 11.146169, 3.280988, obj/third_party/libvpx/libvpx/variance.o
5.407040, 15.328771, 9.921731, obj/components/policy/cloud_policy_proto_generated_compile_proto/cloud_policy.pb.o

Although this is comparing apples to oranges, it would be nice to make clang run faster if there are some low hanging fruits. One possibility might be row_common.o, which is compiled with -march=corei7. Removing the option make it 4x faster.

$ time clang -target x86_64-cros-linux-gnu -O2 -c row_common.cc -march=corei7
real 0m6.920s
user 0m6.900s
sys 0m0.022s

$ time clang -target x86_64-cros-linux-gnu -O2 -c row_common.cc
real 0m1.565s
user 0m1.526s
sys 0m0.037s

Top 10 time-consuming functions:

Overhead Command Shared Object Symbol

........ ................ ................ .................................

12.08%  clang-4.0         clang-4.0         [.] combineX86ShufflesRecursively                               
 8.81%  clang-4.0         libc-2.23.so      [.] malloc                                                      
 8.05%  clang-4.0         clang-4.0         [.] llvm::extractConstantMask                                   
 6.86%  clang-4.0         clang-4.0         [.] llvm::APInt::shlSlowCase                                    
 4.84%  clang-4.0         clang-4.0         [.] llvm::APInt::lshr                                           
 2.13%  clang-4.0         libstdc++.so.6.0  [.] operator new                                                
 1.92%  clang-4.0         clang-4.0         [.] peekThroughBitcasts                                         
 1.63%  clang-4.0         clang-4.0         [.] llvm::DecodePSHUFBMask                                      
 1.61%  clang-4.0         clang-4.0         [.] llvm::APInt::operator|=                                     
 1.39%  clang-4.0         clang-4.0         [.] llvm::APInt::initSlowCase

[llvmbot]

assigned to @RKSimon

[RKSimon]

I've take a look at what is killing us in APInt bit manipulation of multi-word (> 64 bit) values.

The 3 patterns in particular are:

Bits |= APInt::getBitsSet(SizeInBits, Lo, Hi);

Bits |= Sub.zextOrTrunc(SizeInBits).shl(Lo);

APInt Sub = Bits.lshr(Lo).zextOrTrunc(SubSizeInBits);

with 'Bits' typically 128/256/512 bits in size and 'Sub' always 64 bits or less.

With this in mind, APInt methods that would speed things up would be along the lines of (along with some questions on the exact behaviour we want):

// Set all bits to true from loBit (inclusive) to hiBit (exclusive).
// Wraparound behaviour is identical to APInt::getBitsSet.
void APInt::setBits(unsigned loBit, unsigned hiBit);

// Insert bits with Sub starting from offset.
// ?? Wraparound behaviour
// ?? Replace existing bits or OR with them?
void APInt::insertBits(APInt Sub, unsigned offset);

// Extract bits from loBit (inclusive) to hiBit (exclusive).
// ?? Wraparound behaviour
APInt APInt::extractBits(unsigned loBit, unsigned hiBit);

[zmodem]

+Simon and Chandler for combineX86ShufflesRecursively

[RKSimon]

With this in mind, APInt methods that would speed things up would be along
the lines of (along with some questions on the exact behaviour we want):

// Set all bits to true from loBit (inclusive) to hiBit (exclusive).
// Wraparound behaviour is identical to APInt::getBitsSet.
void APInt::setBits(unsigned loBit, unsigned hiBit);

https://reviews.llvm.org/D30265 proposes APInt::setBits to try and avoid:

Bits |= APInt::getBitsSet(SizeInBits, Lo, Hi);

[rotateright]

I really appreciate the self-contained test case. :)

Here's a proposal for a small fix to the IR optimizer (opt):
https://reviews.llvm.org/D30270

For this test, that will just be noise until the backend is sped-up, but there have been recent complaints (but no test cases that I know of) about InstCombine / InstSimplify compile-time in general, so hopefully that makes all optimized compiles a little faster.

[rotateright]

I really appreciate the self-contained test case. :)

Here's a proposal for a small fix to the IR optimizer (opt):
https://reviews.llvm.org/D30270

https://reviews.llvm.org/rL295898
https://reviews.llvm.org/rL296129

[rnk]

Can we mark fixed?

[rotateright]

My changes should have close to no difference on this benchmark, so you can safely ignore those.

Here's what I see running on 4GHz Haswell with a recent release build of clang:

$ ./clang -v
clang version 5.0.0 (trunk 296695)
Target: x86_64-apple-darwin16.4.0

$ time ./clang -target x86_64-cros-linux-gnu -O2 row_common.cc -c -o /dev/null -march=corei7

real 0m6.254s
user 0m6.215s
sys 0m0.016s
bin $ time ./clang -target x86_64-cros-linux-gnu -O2 row_common.cc -c -o /dev/null

real 0m1.292s
user 0m1.263s
sys 0m0.014s
bin $ time ./clang -target x86_64-cros-linux-gnu -O2 row_common.cc -c -o /dev/null -mavx

real 0m1.138s
user 0m1.110s
sys 0m0.012s

---

So I'd say there's still a lot of potential to improve the backend. We're still combining shuffles recursively for an SSE3 or SSE4 target for a very long time.

[RKSimon]

I still want to get the APInt::insertBits implementation done after which I need to start trying to break down where the time is going in combineX86ShufflesRecursively

[RKSimon]

I've added a patch for APInt::insertBits - https://reviews.llvm.org/D30780

[RKSimon]

combineX86ShufflesRecursively - I've avoided some APInt malloc calls in rL297267

[RKSimon]

Improved llvm::extractConstantMask in rL297381

[rotateright]

combineX86ShufflesRecursively - I've avoided some APInt malloc calls in
rL297267

It's getting better. I'm down to 5.05 secs (23% faster) than the corei7 run in comment 7.

[RKSimon]

Committed the APInts::insertBits support at rL297458

[rotateright]

Committed the APInts::insertBits support at rL297458

Another good improvement on my build machine: 4.35 sec (16% faster than the last check).

The base (not corei7) build is still running 1.25 sec, so there's still a lot of room for improvement.

An Instruments profile on macOS says that the hottest path is 8-deep on combineX86ShufflesRecursively(). Can we limit the recursion and still catch most shuffle optimizations? combineShuffle() is ~80% of the total compile time.