PPC codgen issue on Windows

[ltertan]

Compile options:

--target=ppc64 -mcpu=a2 -mhard-float

Test-case:

#include <stdio.h>
int myTest()
{
    unsigned long long value0, value1;
    value0 = 0x1234;
    value1 = 0x0000800000000000UL;
    
    if (value0 < 0x0000800000000000UL)
        printf ("correct\n");
    else
        printf ("incorrect\n");
    
    if (value0 < value1)
        printf ("correct\n");
    else
        printf ("incorrect\n");
    
    return 0;
} 

Description:

On a Windows host, the first if block gets compiled in a such a manner that we would print "incorrect".

[llvmbot]

@llvm/issue-subscribers-backend-powerpc

[nemanjai]

I'm not sure I can help here. I have no idea why you are getting different behaviour on a Windows host. What IR does it produce?

[ltertan]

I think this might be a front-end issue. On Linux the 0x0000800000000000UL literal in the if is treated as a 8 byte value, while on Windows as a 4 byte value.

[nemanjai]

Oh, right. In 32-bit mode, presumably long is 32-bits wide. Presumably it will work if you change it to 0x0000800000000000ULL

[ltertan]

But the Windows host is 64-bit, so is the ULL over UL necessary?

[nemanjai]

Can you just print the value to see if it is truncated as well as print sizeof(long)? That should tell you what size the values are and whether the literal is truncated.

[ltertan]

I ran a few tests.

Execution of the above Test-case prints:

incorrect
correct

I was able to run a few more tests:

printf ("sizeof(long) = %d\n", sizeof(long));
printf ("0x0000800000000000UL: %lu\n", 0x0000800000000000UL);
printf ("value1: %lu\n", value1);

This outputs:

sizeof(long) = 8
0x0000800000000000UL: 140737488355328                                           
value1: 140737488355328

Generated assembly-code (for the initial Test-case, without the 3 printf() calls above):

0000000000000010 <myTest>:
  10:   3c 4c 00 00     addis   r2,r12,0
  14:   38 42 00 00     addi    r2,r2,0
  18:   7c 08 02 a6     mflr    r0
  1c:   fb e1 ff f8     std     r31,-8(r1)
  20:   f8 01 00 10     std     r0,16(r1)
  24:   f8 21 ff 81     stdu    r1,-128(r1)
  28:   7c 3f 0b 78     mr      r31,r1
  2c:   38 60 12 34     li      r3,4660
  30:   f8 7f 00 70     std     r3,112(r31)
  34:   38 60 00 01     li      r3,1
  38:   78 63 7c 06     rldicr  r3,r3,47,16
  3c:   f8 7f 00 68     std     r3,104(r31)
  40:   e8 7f 00 70     ld      r3,112(r31)
  44:   28 23 00 00     cmpldi  r3,0
  48:   40 80 00 18     bge     60 <myTest+0x50>
  4c:   3c 62 00 00     addis   r3,r2,0
  50:   38 63 00 00     addi    r3,r3,0
  54:   48 00 00 01     bl      54 <myTest+0x44>
  58:   60 00 00 00     nop
  5c:   48 00 00 14     b       70 <myTest+0x60>
  60:   3c 62 00 00     addis   r3,r2,0
  64:   38 63 00 00     addi    r3,r3,0
  68:   48 00 00 01     bl      68 <myTest+0x58>
  6c:   60 00 00 00     nop
  70:   e8 7f 00 70     ld      r3,112(r31)
  74:   e8 9f 00 68     ld      r4,104(r31)
  78:   7c 23 20 40     cmpld   r3,r4
  7c:   40 80 00 18     bge     94 <myTest+0x84>
  80:   3c 62 00 00     addis   r3,r2,0
  84:   38 63 00 00     addi    r3,r3,0
  88:   48 00 00 01     bl      88 <myTest+0x78>
  8c:   60 00 00 00     nop
  90:   48 00 00 14     b       a4 <myTest+0x94>
  94:   3c 62 00 00     addis   r3,r2,0
  98:   38 63 00 00     addi    r3,r3,0
  9c:   48 00 00 01     bl      9c <myTest+0x8c>
  a0:   60 00 00 00     nop
  a4:   38 60 00 00     li      r3,0
  a8:   38 21 00 80     addi    r1,r1,128
  ac:   e8 01 00 10     ld      r0,16(r1)
  b0:   eb e1 ff f8     ld      r31,-8(r1)
  b4:   7c 08 03 a6     mtlr    r0
  b8:   4e 80 00 20     blr

The same code, generated on Linux using the same options:

0000000000000010 <myTest>:
  10:   3c 4c 00 00     addis   r2,r12,0
  14:   38 42 00 00     addi    r2,r2,0
  18:   7c 08 02 a6     mflr    r0
  1c:   fb e1 ff f8     std     r31,-8(r1)
  20:   f8 01 00 10     std     r0,16(r1)
  24:   f8 21 ff c1     stdu    r1,-64(r1)
  28:   7c 3f 0b 78     mr      r31,r1
  2c:   38 60 12 34     li      r3,4660
  30:   f8 7f 00 30     std     r3,48(r31)
  34:   38 60 00 01     li      r3,1
  38:   78 63 7c 06     rldicr  r3,r3,47,16
  3c:   f8 7f 00 28     std     r3,40(r31)
  40:   e8 7f 00 30     ld      r3,48(r31)
  44:   38 80 00 01     li      r4,1
  48:   78 84 7c 06     rldicr  r4,r4,47,16
  4c:   7c 23 20 40     cmpld   r3,r4
  50:   40 80 00 20     bge     70 <myTest+0x60>
  54:   3c 62 00 00     addis   r3,r2,0
  58:   38 63 00 00     addi    r3,r3,0
  5c:   3c 62 00 00     addis   r3,r2,0
  60:   38 63 00 00     addi    r3,r3,0
  64:   48 00 00 01     bl      64 <myTest+0x54>
  68:   60 00 00 00     nop
  6c:   48 00 00 1c     b       88 <myTest+0x78>
  70:   3c 62 00 00     addis   r3,r2,0
  74:   38 63 00 00     addi    r3,r3,0
  78:   3c 62 00 00     addis   r3,r2,0
  7c:   38 63 00 00     addi    r3,r3,0
  80:   48 00 00 01     bl      80 <myTest+0x70>
  84:   60 00 00 00     nop
  88:   e8 7f 00 30     ld      r3,48(r31)
  8c:   e8 9f 00 28     ld      r4,40(r31)
  90:   7c 23 20 40     cmpld   r3,r4
  94:   40 80 00 20     bge     b4 <myTest+0xa4>
  98:   3c 62 00 00     addis   r3,r2,0
  9c:   38 63 00 00     addi    r3,r3,0
  a0:   3c 62 00 00     addis   r3,r2,0
  a4:   38 63 00 00     addi    r3,r3,0
  a8:   48 00 00 01     bl      a8 <myTest+0x98>
  ac:   60 00 00 00     nop
  b0:   48 00 00 1c     b       cc <myTest+0xbc>
  b4:   3c 62 00 00     addis   r3,r2,0
  b8:   38 63 00 00     addi    r3,r3,0
  bc:   3c 62 00 00     addis   r3,r2,0
  c0:   38 63 00 00     addi    r3,r3,0
  c4:   48 00 00 01     bl      c4 <myTest+0xb4>
  c8:   60 00 00 00     nop
  cc:   38 60 00 00     li      r3,0
  d0:   38 21 00 40     addi    r1,r1,64
  d4:   e8 01 00 10     ld      r0,16(r1)
  d8:   eb e1 ff f8     ld      r31,-8(r1)
  dc:   7c 08 03 a6     mtlr    r0
  e0:   4e 80 00 20     blr

[efriedma-quic]

Could you clarify exactly what compiler binaries you're using on "Windows" vs. "Linux"? It's possible to have bugs where the compiler output depends on the endianness or pointer size of the host. (It could narrow down the issue if you try 32-bit Windows vs. 64-bit Windows.)

[nemanjai]

Also, perhaps provide the full command line invocation on both systems as well as what clang --version outputs. It might help bisect the problem to see if the IR changes between the two systems (i.e. is the problem in the back end or before). Does anything change if you compile with -O2 (i.e. perhaps we have an issue in FastISel that is somehow dependent on the host)?

It is clear from the assembly that the build on Windows compares 0x1234 against 0:

  2c:   38 60 12 34     li      r3,4660 # materialize 0x1234
  30:   f8 7f 00 70     std     r3,112(r31) # store it to the stack
...
  40:   e8 7f 00 70     ld      r3,112(r31) # load it back from the stack
  44:   28 23 00 00     cmpldi  r3,0 # compare it against zero

whereas the assembly in the build on Linux compares it against the expected value:

  2c:   38 60 12 34     li      r3,4660 # materialize 0x1234
  30:   f8 7f 00 30     std     r3,48(r31) # store it to the stack
...
  40:   e8 7f 00 30     ld      r3,48(r31) # load it back from the stack 
  44:   38 80 00 01     li      r4,1 # materialize 1
  48:   78 84 7c 06     rldicr  r4,r4,47,16 # shift it left 47 bits (i.e. materialize 1 << 47, which is 0x0000800000000000UL)
  4c:   7c 23 20 40     cmpld   r3,r4 # compare those two values

[ltertan]

Command line invocation for Windows 10 64-bit, using official LLVM 13.0.1:

C:\wr>clang --target=ppc64 -mcpu=a2 -c code.c -###

clang version 13.0.1
Target: ppc64
Thread model: posix
InstalledDir: C:\Program Files\LLVM\bin
 (in-process)
 "C:\\Program Files\\LLVM\\bin\\clang.exe" "-cc1" "-triple" "ppc64" "-emit-obj" "-mrelax-all" "--mrelax-relocations" "-disable-free" "-disable-llvm-verifier" "-discard-value-names" "-main-file-name" "code.c" "-mrelocation-model" "static" "-mframe-pointer=all" "-fmath-errno" "-fno-rounding-math" "-mconstructor-aliases" "-munwind-tables" "-target-cpu" "a2" "-mfloat-abi" "hard" "-target-abi" "elfv1" "-debugger-tuning=gdb" "-fcoverage-compilation-dir=C:\\wr" "-resource-dir" "C:\\Program Files\\LLVM\\lib\\clang\\13.0.1" "-fdebug-compilation-dir=C:\\wr" "-ferror-limit" "19" "-fmessage-length=102" "-fno-signedchar" "-fgnuc-version=4.2.1" "-fcolor-diagnostics" "-faddrsig" "-D__GCC_HAVE_DWARF2_CFI_ASM=1" "-o" "code.o" "-x" "c" "code.c"

Against command line invocation on Ubuntu 18.04, using the official LLVM 13.0.1:

$ clang --target=ppc64 -mcpu=a2 -c codgen.c  -###

clang version 13.0.1
Target: ppc64
Thread model: posix
InstalledDir: /home/ltertan/llvmorg/clang+llvm-13.0.1-x86_64-linux-gnu-ubuntu-18.04/bin
 (in-process)
 "/home/ltertan/llvmorg/clang+llvm-13.0.1-x86_64-linux-gnu-ubuntu-18.04/bin/clang-13" "-cc1" "-triple" "ppc64" "-emit-obj" "-mrelax-all" "--mrelax-relocations" "-disable-free" "-disable-llvm-verifier" "-discard-value-names" "-main-file-name" "codgen.c" "-mrelocation-model" "static" "-mframe-pointer=all" "-fmath-errno" "-fno-rounding-math" "-mconstructor-aliases" "-munwind-tables" "-target-cpu" "a2" "-mfloat-abi" "hard" "-target-abi" "elfv1" "-debugger-tuning=gdb" "-fcoverage-compilation-dir=/home/ltertan/workspace/tmp" "-resource-dir" "/home/ltertan/llvmorg/clang+llvm-13.0.1-x86_64-linux-gnu-ubuntu-18.04/lib/clang/13.0.1" "-fdebug-compilation-dir=/home/ltertan/workspace/tmp" "-ferror-limit" "19" "-fno-signed-char" "-fgnuc-version=4.2.1" "-fcolor-diagnostics" "-faddrsig" "-D__GCC_HAVE_DWARF2_CFI_ASM=1" "-o" "codgen.o" "-x" "c" "codgen.c"

I don't see anything significant changed in between the two calls.

As per the 32-bit vs 64-bit, I was only able to test and cofirm the same behaviour when running the 32-bit and 64-bit binaries on a 64-bit Windows machine, but this shouldn't matter much. I have yet to test the 32-bit LLVM on a 32-bit machine, @efriedma-quic.

I've tried compiling using -O2 and -O and the problem went away as the code was massively optimized/reduced, and the if statements are gone, with direct calls to printf() instead. The optimization in this case produces the same (correct) calls to printf() on both Linux and Windows.

Hey @nemanjai,

With -mllvm -opt-bisect-limit=1 I am still able to reproduce the problem, same as with -O2 -mllvm -opt-bisect-limit=1, so maybe this looks like a problem in the parser/front-end?

[nemanjai]

Have you looked at the IR produced on the two systems (without optimizations)? I would imagine that the IR would have to be different since the back end isn't doing anything weird.

[ltertan]

surprisingly enough, it's identical:

--- codgen.ll   2022-07-13 14:45:45.877461939 +0300
+++ win64.ll    2022-07-13 14:45:35.317361846 +0300
@@ -1,5 +1,5 @@
-; ModuleID = 'codgen.c'
-source_filename = "codgen.c"
+; ModuleID = 'code.c'
+source_filename = "code.c"
 target datalayout = "E-m:e-i64:64-n32:64"
 target triple = "ppc64"

As expected, the generated assembly file by running llc outputs different code between the two hosts (Win64/Linux64):

--- codgen.s    2022-07-13 14:51:21.492658906 +0300
+++ win64.s     2022-07-13 14:51:12.148569562 +0300
@@ -1,5 +1,5 @@
        .text
-       .file   "codgen.c"
+       .file   "code.c"
        .globl  myTest                          # -- Begin function myTest
        .p2align        4
        .type   myTest,@function
@@ -28,9 +28,7 @@
        sldi 3, 3, 47
        std 3, 120(31)
        ld 3, 128(31)
-       li 4, 1
-       sldi 4, 4, 47
-       cmpld   3, 4
+       cmpldi  3, 0
        bge 0, .LBB0_2
 # %bb.1:
        li 3, 0
➜ tmp $ 

This means there's something else in the back-end that does this transformation, right?