Failed to cross compile with CLANG

[maroviher]

I am trying to cross compile for android AARCH64.

./configure CC=aarch64-linux-android-clang CXX=aarch64-linux-android-clang++ \
LDFLAGS=-L/tmp/openssl-android/lib LD=aarch64-linux-android-ld \
--host=aarch64-linux --with-openssl=/tmp/openssl-android --enable-libnl

make:

  CC       libaircrack_crypto_arm_neon_la-simd-intrinsics.lo
simd-intrinsics.c:273:2: error: argument should be a value from 0 to 31
        MD5_STEP(MD5_F, a, b, c, d, 0, 0xd76aa478, 7)
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
simd-intrinsics.c:125:10: note: expanded from macro 'MD5_STEP'
                a[i] = vroti_epi32(a[i], (s));                                         \
                       ^~~~~~~~~~~~~~~~~~~~~~
./pseudo_intrinsics.h:97:20: note: expanded from macro 'vroti_epi32'
                   : vsriq_n_u32(vshlq_n_u32(x, 32 + (i)), x, -(i)))
                     ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/toolchains/android_6/lib64/clang/6.0.2/include/arm_neon.h:22971:24: note: expanded from macro 'vshlq_n_u32'
  __ret = (uint32x4_t) __builtin_neon_vshlq_n_v((int8x16_t)__s0, __p1, 50); \
                       ^
/opt/toolchains/android_6/lib64/clang/6.0.2/include/arm_neon.h:24656:21: note: expanded from macro 'vsriq_n_u32'
  uint32x4_t __s0 = __p0; \
                    ^~~~

Any ideas?

[jbenden]

Please post theconfig.log and ./configure output. Also, were there any CFLAGS used?

Thanks,
-Joe

[maroviher]

No CFLAGS were set. Here are 2 files, as you metioned:

configure.output.txt
config.log

[jbenden]

Thanks!

An important observation is the compiler accepts the NEON flag, but issues a warning on it:

clang60++: warning: argument unused during compilation: '-mfpu=neon' [-Wunused-command-line-argument]

Ideally, it would have failed this test.

I'll have to gain access to a suitable machine and see what I can do.

Best regards,
-Joe

[jbenden]

I have committed a change that forces Clang to not report success for this test. However, I was using a very, very old version of Clang. Unfortunately, it went on to report other errors.

Therefore, this patch may or may not completely fix the reported issue.

FWIW: On the other hand, it is known that GCC compiles this all just fine.

[maroviher]

But GCC is deprecated in Android NDK:
android/ndk#26

[jbenden]

First see if the change makes a difference, if not we'll figure out the next step.

[ChrisLundquist]

FWIW: here is a hack to run foreign architectures on x86.
https://rwmj.wordpress.com/2013/12/22/how-to-run-aarch64-binaries-on-an-x86-64-host-using-qemu-userspace-emulation/

Basically, it registers QEMU as the handler when it sees a matching binary file. One caveat is that it will have to be statically linked (since naturally you won't be able to load the foreign libs at runtime)

[Grimler91]

Hi!
I'm also looking into compiling aircrack-ng 1.3 for android arm and aarch64 (for the terminal emulator termux, with clang/android ndk r18) and I get the same error for both architectures. Aircrack-ng 1.2-rc4 compiled without problem so this might be related to the new build system (in some way).

Building on device works on arm but not aarch64, the aarch64 build fails with the same error as when cross compiling, I would guess that NEON_TRUE='#' leads to the build succeeding while NEON_FALSE='#' leads to the build failing.

Applying the patch in 1e5daa4 silences the configure warnings but it doesn't seem to give a different result. Here's the logs from configure from aarch64, from arm and from cross-compiling targeting aarch64 (with 1e5daa4 applied):
aarch64_config.log
aarch64_configure.log
arm_config.log
arm_configure.log
cross_config.log
cross_configure.log

[ChrisLundquist]

1.2 probably works because the crypto engine stuff isn't there.
hashing got notably faster in 1.3.

These particular lines come from JtR and it looks like this issue has been hit before:
https://www.openwall.com/lists/john-dev/2015/08/06/3

openwall/john#3284

[jbenden]

I now know why this code does not compile; however, I do not have an idea on how to "quickly" fix it.

The snippet complaining is:

#define vroti_epi32(x, i)                                         \
        (i < 0 ? vsliq_n_u32(vshrq_n_u32(x, 32 - i), x, i)        \
               : vsriq_n_u32(vshlq_n_u32(x, 32 + i), x, -i))

These NEON instructions are a variant that require constant values for certain parameters; due to them being directly encoded in the output opcode. While this is not the real problem, the problem is Clang is "compiling" both side of the test expression. In turn, the failing side fails to compile because it's math on the constant parameter DOES exceed the permitted value. However, every other compiler simply takes the working expression side and discards the other.

Therefore, there is no easy fix currently, and this honestly is a variation in Clang - which means it might very well be a compiler bug. While having them fix this would be certainly useful, it does not fix the immediate needs of those wishing to use the Android NDK.

Thoughts, suggestions, and ideas are welcomed,
-Joe

[jbenden]

#include <arm_neon.h>

#define vroti_epi32(x, i)                                         \
        (i < 0 ? vsliq_n_u32(vshrq_n_u32(x, 32 - i), x, i)        \
               : vsriq_n_u32(vshlq_n_u32(x, 32 + i), x, -i))

int main()
{
	uint32x4_t x = vdupq_n_u32(42);
	uint32x4_t value = vroti_epi32(x, 12);

	return 0;
}

jbenden@gcc113:~$ 

jbenden@gcc113:~$ clang-3.6 -o example example.c
example.c:10:21: error: argument should be a value from 0 to 31
        uint32x4_t value = vroti_epi32(x, 12);
                           ^
example.c:5:30: note: expanded from macro 'vroti_epi32'
               : vsriq_n_u32(vshlq_n_u32(x, 32 + i), x, -i))
                             ^
/usr/lib/llvm-3.6/bin/../lib/clang/3.6.0/include/arm_neon.h:23019:24: note: expanded from macro 'vshlq_n_u32'
  __ret = (uint32x4_t) __builtin_neon_vshlq_n_v((int8x16_t)__s0, __p1, 50); \
                       ^
/usr/lib/llvm-3.6/bin/../lib/clang/3.6.0/include/arm_neon.h:24704:21: note: expanded from macro 'vsriq_n_u32'
  uint32x4_t __s0 = __p0; \
                    ^
example.c:10:21: error: argument should be a value from 1 to 32
        uint32x4_t value = vroti_epi32(x, 12);
                           ^
example.c:5:18: note: expanded from macro 'vroti_epi32'
               : vsriq_n_u32(vshlq_n_u32(x, 32 + i), x, -i))
                 ^
/usr/lib/llvm-3.6/bin/../lib/clang/3.6.0/include/arm_neon.h:24707:24: note: expanded from macro 'vsriq_n_u32'
  __ret = (uint32x4_t) __builtin_neon_vsriq_n_v((int8x16_t)__s0, (int8x16_t)__s1, __p2, 50); \
                       ^
2 errors generated.
jbenden@gcc113:~$ cat example.c

[jbenden]

One idea for a fix is to create each of these intrinsic instructions as inline static functions; whereby, the inner logic of each function is a giant switch statement for all numbers from 0-31 and 1-32.

In doing so, a similar, but different macro using these new functions would not complain.

Yucky!

[jbenden]

Some testing shows this idea sort of works, but requires a lot of "work-around" functions. It also severely impacts overall performance.

I recommend using the generic + static aircrack-ng build, for now. ./configure --enable-static --disable-shared

Thanks,
-Joe

[jbenden]

https://bugs.llvm.org/show_bug.cgi?id=39087

[ghost]

@jbenden the issue on bugs.llvm.org is still open and someone is commenting on it

[jbenden]

@etmatrix There are CLANG bugs that prevent this, right now...

[ghost]

I tried to compile version 1.5.2 on debian 7 wheezy, I got the same issue. Debian 7 use gcc 4.6. So is not a clang bug but a missing feature. Feature introduced in an unknow version of gcc.

make[2]: Entering directory `/root/aircrack-ng/src/aircrack-crypto'
  CC     libaircrack_crypto_la-memory.lo
  CC     libaircrack_crypto_la-sha1-git.lo
  CC     libaircrack_crypto_la-wpapsk.lo
  CC     libaircrack_crypto_la-crypto_engine.lo
  CCLD   libaircrack-crypto.la
  CC     libaircrack_crypto_arm_neon_la-memory.lo
  CC     libaircrack_crypto_arm_neon_la-sha1-git.lo
  CC     libaircrack_crypto_arm_neon_la-simd-intrinsics.lo
/tmp/ccjENqci.s: Assembler messages:
/tmp/ccjENqci.s:22886: Error: co-processor offset out of range
/tmp/ccjENqci.s:22898: Error: co-processor offset out of range
make[2]: *** [libaircrack_crypto_arm_neon_la-simd-intrinsics.lo] Error 1
make[2]: Leaving directory `/root/aircrack-ng/src/aircrack-crypto'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/root/aircrack-ng/src'
make: *** [all-recursive] Error 1

My scope is running aircrack-ng on an old nokia n900 and smartphone with android 8, and the only way is debian chroot.

[areese]

For AppleSilicon look at John the Ripper.
Specifically: openwall/john@c9825e6

[ryandesign]

For AppleSilicon look at John the Ripper.
Specifically: openwall/john@c9825e6

The equivalent fix was committed to aircrack-ng in a4cdb89 and is included in version 1.7.