hyper scan library crash #20
Comments
|
Yes, the difference is that the two machines support different instruction set extensions. The machine you used to compile the HS library uses a 4th Gen i7 (which goes by the codename Haswell). This machine supports AVX2 and BMI1/2 instructions, and when we compile with To fix this, you'll either need to compile Hyperscan on the Xeon, or configure the build on the first machine by passing in the correct |
|
hi, mdb256, I'm one guy with TidyHuang. I have a question, Where do you get the gen and supports of CPU type? When we use march flags, we will use the CPU type of running machine, is it right? Have we one method to let program run all the Intel X86_64 CPU? I don't get one through gcc manpage. Our gcc version is: Through "man gcc", I got this: Thanks a lot. |
|
Yes, you should choose the architecture of the machine that you plan to run Hyperscan on. The list from GCC 4.8 is somewhat confusing - and I should note it has changed in newer versions of GCC - but the minimum feature set required for Hyperscan is The Xeon v2 that was mentioned earlier is covered by The Haswell that you first built Hyperscan on would be using the feature sets from |
|
OK, Thanks for soon reply again! It's so clear now. |
|
Hi Matt, My questions: -Tidy |
|
I'm not quite sure I understand. Firstly I suspect you mean " If you compile Hyperscan with |
|
Thanks Matt, there is a typo for mtune=generic. In theory�$B!$�(B there should be no existing library with march=native. two if us have done such testing. I'll use a clean VM to test and sperate depend lib one by one. |
|
Can hyperscan detect CPU features in runtime? When I compile hyperscan with |
|
The Compiling the Hyperscan lib with |
|
@mdb256, is it possible to enable Note: there are actually more than 2 instruction sets, so replace 2 with actual number. Note 2: it would be even better if hyperscan had cross-platform bytecode. |
|
Something like this: diff --git a/CMakeLists.txt b/CMakeLists.txt
index 6710979..2a398e4 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -184,10 +184,10 @@ else()
if (NOT CMAKE_C_FLAGS MATCHES .*march.*)
message(STATUS "Building for current host CPU")
- set(EXTRA_C_FLAGS "${EXTRA_C_FLAGS} -march=native -mtune=native")
+ set(EXTRA_C_FLAGS "${EXTRA_C_FLAGS} -mssse3")
endif()
if (NOT CMAKE_CXX_FLAGS MATCHES .*march.*)
- set(EXTRA_CXX_FLAGS "${EXTRA_CXX_FLAGS} -march=native -mtune=native")
+ set(EXTRA_CXX_FLAGS "${EXTRA_CXX_FLAGS} -mssse3")
endif()
if(CMAKE_COMPILER_IS_GNUCC)
@@ -540,13 +540,16 @@ set (hs_exec_SRCS
src/database.h
)
-if (HAVE_AVX2)
set (hs_exec_SRCS
${hs_exec_SRCS}
src/fdr/teddy_avx2.c
src/util/masked_move.c
)
-endif ()
+ set_source_files_properties(
+ src/fdr/teddy_avx2.c
+ src/util/masked_move.c
+ PROPERTIES COMPILE_FLAGS -mavx2
+ )
SET (hs_SRCS
|
|
Unfortunately it isn't as simple as just building some of the files with avx2 - those two files are only required for avx2 builds, but there are many more places we use avx2 instructions where they are available. Similarly in the Hyperscan lib we use a mix of other microarch additions where we can, like sse4.2 (crc32), popcnt, bmi2 (pext, pdep), and more. We have looked at building a "fat binary", or as you say a universal lib that supports as many different microarchitectures as required - but it is going to take some time, and has portability problems. Plus we need to be careful about mixing SSE and AVX instructions, as switching between them can incur expensive performance penalties. |
|
which flag make it executable for all range of hardware's ? |
|
@sadegh01, |
|
As the hs core is written in C++ (as far as I can see), wouldn't use of function multiversioning https://gcc.gnu.org/wiki/FunctionMultiVersioning be applicable here? |
|
FMV seems to be a popular topic lately. I spent a while trying to make it work for Hyperscan, but it wasn't the right fit. We have a working version of the fat runtime working that I mentioned above - it is still a bit experimental, but I'll be pushing the commits soon. It works by building n-copies of the runtime code (the C, not the C++) and uses the indirect function attribute to dispatch the right API function based on what the host platform supports. |
|
Hyperscan v4.4 includes the fat runtime work for Linux, and this issue is becoming a collection of somewhat related items. I'm going to close this issue, but if please open a new issue or contact us directly if there are any problems. |
Ci ubuntu 22
Hello,
I have been developing an app with very simple function to compile a pattern and use the pattern to match which needs use Hyperscan library:
After compiling the hyperscan library with following options: -DCMAKE_BUILD_TYPE=debug -DBOOST_ROOT=${BOOST_ROOT} -DBUILD_SHARED_LIBS=1 -DCMAKE_INSTALL_PREFIX=${USR_LIB_PATH}.
The App runs as expected at my compiling machine[model name : Intel(R) Core(TM) i7-4650U CPU @ 1.70GHz], and then I copied my app and libhs.so to another machine [model name : Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz](Same X86 64bit machine).
however it crashed every time at pattern load function when I execute the app.
The compiling machine and running machine has difference cpuinfo, not sure it has relationship with this.
Could some one can help me out. Thanks.
Here are the crash info:
Compiling machine cpuinfo:
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 69
model name : Intel(R) Core(TM) i7-4650U CPU @ 1.70GHz
stepping : 1
microcode : 0x17
cpu MHz : 2300.092
cache size : 4096 KB
physical id : 0
siblings : 1
core id : 0
cpu cores : 1
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts nopl xtopology tsc_reliable nonstop_tsc aperfmperf eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm ida arat epb pln pts dtherm fsgsbase tsc_adjust bmi1 avx2 smep bmi2 invpcid xsaveopt
bogomips : 4600.18
clflush size : 64
cache_alignment : 64
address sizes : 42 bits physical, 48 bits virtual
power management:
Running machine cpuinfo:
cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 62
model name : Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
stepping : 4
microcode : 0x415
cpu MHz : 2494.028
cache size : 25600 KB
physical id : 0
siblings : 1
core id : 0
cpu cores : 1
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology eagerfpu pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm fsgsbase smep erms xsaveopt
bugs :
bogomips : 4988.05
clflush size : 64
cache_alignment : 64
address sizes : 46 bits physical, 48 bits virtual
power management:
-Tidy
The text was updated successfully, but these errors were encountered: