Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Undeclared HWCAP_CRC32 compilation error in ARMv8 (Jetson TX2) #26458

Closed
cdsousa opened this issue Mar 14, 2018 · 18 comments · Fixed by #30324
Closed

Undeclared HWCAP_CRC32 compilation error in ARMv8 (Jetson TX2) #26458

cdsousa opened this issue Mar 14, 2018 · 18 comments · Fixed by #30324
Labels
building Build system, or building Julia or its dependencies system:arm ARMv7 and AArch64

Comments

@cdsousa
Copy link
Contributor

cdsousa commented Mar 14, 2018

I'm getting this error:
/home/nvidia/julia/src/crc32c.c:336:17: error: 'HWCAP_CRC32' undeclared (first use in this function)
when compiling Julia master, at a0180d9, in a Jetson TX2 (ARMv8).
Two months older versions used to compile flawlessly.

@vchuravy
Copy link
Member

The last change to that file was in October. What C/C++ compiler are you using, and did you maybe update your compiler in the mean time?

@vchuravy vchuravy added system:arm ARMv7 and AArch64 building Build system, or building Julia or its dependencies labels Mar 14, 2018
@stevengj
Copy link
Member

stevengj commented Mar 14, 2018

This line was added in #22385; HWCAP_CRC32 should be defined in your compiler's hwcap.h file

@yuyichao
Copy link
Contributor

Not sure what changed but try adding # include <hwcap.h> here ?

@yuyichao
Copy link
Contributor

Actually that shouldn't be needed I haven't found the aarch64 definition yet but According to https://github.molgen.mpg.de/git-mirror/glibc/blob/20003c49884422da7ffbc459cdeee768a6fee07b/sysdeps/unix/sysv/linux/arm/bits/hwcap.h#L20 including auxv.h is the correct way.

@yuyichao
Copy link
Contributor

Found it. It was added 2 years ago in bminor/glibc@26c2910 and the usage we have seems correct. What's your glibc version and what's your bits/hwcap.h ? (should be /usr/include/bits/hwcap.h)

@cdsousa
Copy link
Contributor Author

cdsousa commented Mar 14, 2018

Thanks.
Running the gcc command with -M:

nvidia@tegra-ubuntu:~/julia/src$ gcc -march=armv8-a -fasynchronous-unwind-tables -DJULIA_ENABLE_THREADING -DJULIA_NUM_THREADS=1 -std=gnu99 -pipe -fPIC -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -Wold-style-definition -Wstrict-prototypes -Wc++-compat -O3 -ggdb2 -falign-functions -D_GNU_SOURCE -I. -I/home/nvidia/julia/src -I/home/nvidia/julia/src/flisp -I/home/nvidia/julia/src/support -I/home/nvidia/julia/usr/include -I/home/nvidia/julia/usr/include -DLIBRARY_EXPORTS -I/home/nvidia/julia/deps/valgrind -Wall -Wno-strict-aliasing -fno-omit-frame-pointer -fvisibility=hidden -fno-common -Wpointer-arith -Wundef -DJL_BUILD_ARCH='"aarch64"' -DJL_BUILD_UNAME='"Linux"' -I/home/nvidia/julia/usr/include -DLLVM_SHLIB "-DJL_SYSTEM_IMAGE_PATH=\"../lib/julia/sys.so\"" -DNDEBUG -DJL_NDEBUG -c /home/nvidia/julia/src/crc32c.c -M 

entails:

crc32c.o: /home/nvidia/julia/src/crc32c.c /usr/include/stdc-predef.h \
 /home/nvidia/julia/src/julia.h \
 /home/nvidia/julia/src/support/libsupport.h \
 /home/nvidia/julia/src/support/platform.h /usr/include/stdlib.h \
 /usr/include/features.h /usr/include/aarch64-linux-gnu/sys/cdefs.h \
 /usr/include/aarch64-linux-gnu/bits/wordsize.h \
.....................................
(many other headers)
......................................
 /home/nvidia/julia/usr/include/libunwind-common.h \
 /home/nvidia/julia/usr/include/libunwind-dynamic.h \
 /usr/include/aarch64-linux-gnu/sys/auxv.h /usr/include/elf.h \
 /usr/include/aarch64-linux-gnu/bits/auxv.h \
 /usr/include/aarch64-linux-gnu/bits/hwcap.h \
 /home/nvidia/julia/src/crc32c-tables.c

In fact /usr/include/aarch64-linux-gnu/bits/hwcap.h does not define anything nor includes any other file.

In the system there is a /usr/src/linux-headers-4.4.38-tegra/arch/arm64/include/asm/hwcap.h which includes /usr/src/linux-headers-4.4.38-tegra/arch/arm64/include/uapi/asm/hwcap.h which has a HWCAP_CRC32.

File /usr/include/aarch64-linux-gnu/bits/hwcap.h comes from package libc6-dev which has version 2.23-0ubuntu10.

@cdsousa
Copy link
Contributor Author

cdsousa commented Mar 14, 2018

There is also a /usr/include/aarch64-linux-gnu/asm/hwcap.h file which has HWCAP_CRC32.

@cdsousa
Copy link
Contributor Author

cdsousa commented Mar 14, 2018

/usr/include/aarch64-linux-gnu/bits/hwcap.h
only has

\\ (initial comments)

#ifndef _SYS_AUXV_H
# error "Never include <bits/hwcap.h> directly; use <sys/auxv.h> instead."
#endif

/* No bits defined for this architecture.  */

@yuyichao
Copy link
Contributor

It seems that your glibc is just a little too old. Note that including the kernel header usually isn't a solution due to possible name conflict with glibc ones.

One solution is to do a ifdef check and if not, define our own value. (it's a kernel API so the value is stable).

Another solution is to just use processor.h, which already have this value hard coded.
Does it work if you add #include "processor.h" here and replace the use of HWCAP_CRC32 with JL_AArch64_crc?

@yuyichao
Copy link
Contributor

yuyichao commented Mar 14, 2018

Actually, should be (1 << JL_AArch64_crc).

@cdsousa
Copy link
Contributor Author

cdsousa commented Mar 14, 2018

Ok, guess that what I had compiled two months ago was in fact v0.6.2. That compiles since its source code does not uses HWCAP_CRC32.

Doing

#include "processor.h"
#define HWCAP_CRC32 (1 << JL_AArch64_crc)

works.
But now I'm bumping into another, unrelated, error:

ERROR: Invalid CPU name armv8-a.
Makefile:195: recipe for target '/home/nvidia/julia/usr/lib/julia/basecompiler.ji' failed

@vchuravy
Copy link
Member

JULIA_CPU_TARGET=generic
MARCH=armv8-a

should do the trick.

@yuyichao
Copy link
Contributor

#define HWCAP_CRC32 (1 << JL_AArch64_crc)

Don't do that, replace the actual use.

ERROR: Invalid CPU name armv8-a.

That doesn't make much sense..... There's only one place that can set the flag to cause this which is arg_target_data(const TargetData<feature_sz> &arg, bool require_host) and it can only happen if auto spec = find_cpu(res.name) fails. So you should set a breakpoint there to see why it didn't find the info corresponding to the valid name "armv8-a".

@yuyichao
Copy link
Contributor

yuyichao commented Mar 14, 2018

Ah, I was looking at the wrong CPU spec list. armv8-a is indeed missing from the CPU list for aarch64 (it is in the aarch32 list). It's probably an oversight/something I missed since the plan changes back and forth a few times in that PR.

diff --git a/src/processor_arm.cpp b/src/processor_arm.cpp
index 01b98c2027..2801ea571a 100644
--- a/src/processor_arm.cpp
+++ b/src/processor_arm.cpp
@@ -209,6 +209,7 @@ constexpr auto apple_hurricane = armv8a_crc_crypto;

 static constexpr CPUSpec<CPU, feature_sz> cpus[] = {
     {"generic", CPU::generic, CPU::generic, 0, Feature::generic},
+    {"armv8-a", CPU::generic, CPU::generic, 0, Feature::generic},
     {"armv8.1-a", CPU::armv8_1_a, CPU::generic, 0, Feature::armv8_1a},
     {"armv8.2-a", CPU::armv8_2_a, CPU::generic, 0, Feature::armv8_2a},
     {"armv8.3_a", CPU::armv8_3_a, CPU::generic, 0, Feature::armv8_3a},

Should work.

@yuyichao
Copy link
Contributor

As a side note, armv8-a is supposed to be a valid option since the plan was to support everything reasonable that you can to gcc in -march and -mtune. Reasonable here means architectures that we actually support (so most pre-pentium4 arch are not). Any of those that are not supported should be reported as bugs.

@cdsousa
Copy link
Contributor Author

cdsousa commented Mar 15, 2018

Ok, it is fully working.

Must this issue be closed by considering that it is caused by problems with Jetson TX2 glibc, or there is something that must be done on the Julia side?

@stevengj
Copy link
Member

Wouldn't hurt to update the source to use (1 << JL_AArch64_crc), and it sounds like src/processor_arm.cpp should be updated too.

@simonbyrne
Copy link
Contributor

Should this be a pull request?

staticfloat added a commit that referenced this issue Dec 9, 2018
ararslan pushed a commit that referenced this issue Dec 9, 2018
KristofferC pushed a commit that referenced this issue Dec 10, 2018
Closes #26458

(cherry picked from commit bd21aa7)
KristofferC pushed a commit that referenced this issue Dec 30, 2018
Closes #26458

(cherry picked from commit bd21aa7)
KristofferC pushed a commit that referenced this issue Feb 4, 2019
Closes #26458

(cherry picked from commit bd21aa7)
KristofferC pushed a commit that referenced this issue Feb 11, 2019
Closes #26458

(cherry picked from commit bd21aa7)
KristofferC pushed a commit that referenced this issue Apr 20, 2019
Closes #26458

(cherry picked from commit bd21aa7)
KristofferC pushed a commit that referenced this issue Feb 20, 2020
Closes #26458

(cherry picked from commit bd21aa7)
copybara-service bot pushed a commit to abseil/abseil-cpp that referenced this issue Dec 13, 2024
fix hw crc32 and AES capability check, fix undefined

Imported from GitHub PR #1794

Source and explanation
JuliaLang/julia#26458
memcached/memcached#744

For build for aarch64 on v22_clang-16.0.6-centos7
`
abseil-cpp/absl/crc/internal/cpu_detect.cc:273:20: error: use of undeclared identifier 'HWCAP_CRC32'
  return (hwcaps & HWCAP_CRC32) && (hwcaps & HWCAP_PMULL);
                   ^
abseil-cpp/absl/crc/internal/cpu_detect.cc:273:46: error: use of undeclared identifier 'HWCAP_PMULL'
  return (hwcaps & HWCAP_CRC32) && (hwcaps & HWCAP_PMULL);
`

Merge 3ee325b into 940e0ec

Merging this change closes #1794

COPYBARA_INTEGRATE_REVIEW=#1794 from Dertosh:patch-1 3ee325b
PiperOrigin-RevId: 705936372
Change-Id: Ifebd6d1a854e17acf6cc00bab92053bc0d4c2349
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
building Build system, or building Julia or its dependencies system:arm ARMv7 and AArch64
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants