Can it support avx cpu's older than 10 years old?

[FlowDownTheRiver]

I can't run any model due to my cpu is from before 2013.So I don't have avx2 instructions.Can you please support avx cpus?

[oofaustoo]

If you have a processor that is avx capable but does not have fp16c flags, you will need to disable avx in cflags for it to compile...

I am using some old Xeon E5-2690s which have avx but not fp16c... it compiles w/ just sse3... that being said, dont expect it to be performant in any way... =/

[alextomskras]

Hello, if i can't start on xeon E5-1620v3 it's fron not support fp16c flags ? How I can recompile for god start

[alextomskras]

need disable AVX2 and recompile for work on XEON E5-1650v3

and for work over CMAKE need install VisualStudio2019 community

cmake_minimum_required(VERSION 3.8)
project("alpaca.cpp")

set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED true)
set(CMAKE_C_STANDARD 11)

if (NOT XCODE AND NOT MSVC AND NOT CMAKE_BUILD_TYPE)
    set(CMAKE_BUILD_TYPE Release CACHE STRING "Build type" FORCE)
    set_property(CACHE CMAKE_BUILD_TYPE PROPERTY STRINGS "Debug" "Release" "MinSizeRel" "RelWithDebInfo")
endif()

option(LLAMA_ALL_WARNINGS            "llama: enable all compiler warnings"                   ON)
option(LLAMA_ALL_WARNINGS_3RD_PARTY  "llama: enable all compiler warnings in 3rd party libs" OFF)

option(LLAMA_SANITIZE_THREAD         "llama: enable thread sanitizer"    OFF)
option(LLAMA_SANITIZE_ADDRESS        "llama: enable address sanitizer"   OFF)
option(LLAMA_SANITIZE_UNDEFINED      "llama: enable undefined sanitizer" OFF)
option(LLAMA_NO_AVX2              "llama: disable AVX2" ON)

if (APPLE)
    option(LLAMA_NO_ACCELERATE       "llama: disable Accelerate framework" OFF)
    option(LLAMA_NO_AVX              "llama: disable AVX" OFF)
    option(LLAMA_NO_AVX2             "llama: disable AVX2" OFF)
    option(LLAMA_NO_FMA              "llama: disable FMA" OFF)
endif()

if (NOT MSVC)
    if (LLAMA_SANITIZE_THREAD)
        set(CMAKE_C_FLAGS   "${CMAKE_C_FLAGS}   -fsanitize=thread")
        set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fsanitize=thread")
    endif()

    if (LLAMA_SANITIZE_ADDRESS)
        set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS}     -fsanitize=address -fno-omit-frame-pointer")
        set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fsanitize=address -fno-omit-frame-pointer")
    endif()

    if (LLAMA_SANITIZE_UNDEFINED)
        set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS}     -fsanitize=undefined")
        set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fsanitize=undefined")
    endif()
endif()

if (APPLE AND NOT LLAMA_NO_ACCELERATE)
    find_library(ACCELERATE_FRAMEWORK Accelerate)
    if (ACCELERATE_FRAMEWORK)
        message(STATUS "Accelerate framework found")

        set(LLAMA_EXTRA_LIBS  ${LLAMA_EXTRA_LIBS}  ${ACCELERATE_FRAMEWORK})
        set(LLAMA_EXTRA_FLAGS ${LLAMA_EXTRA_FLAGS} -DGGML_USE_ACCELERATE)
    else()
        message(WARNING "Accelerate framework not found")
    endif()
endif()

if (LLAMA_ALL_WARNINGS)
    if (NOT MSVC)
        set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} \
            -Wall                           \
            -Wextra                         \
            -Wpedantic                      \
            -Wshadow                        \
            -Wcast-qual                     \
            -Wstrict-prototypes             \
            -Wpointer-arith                 \
            -Wno-unused-function            \
        ")
        set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} \
            -Wall                           \
            -Wextra                         \
            -Wpedantic                      \
            -Wcast-qual                     \
        ")
    else()
        # todo : msvc
    endif()
endif()

message(STATUS "CMAKE_SYSTEM_PROCESSOR: ${CMAKE_SYSTEM_PROCESSOR}")

if (${CMAKE_SYSTEM_PROCESSOR} MATCHES "arm" OR ${CMAKE_SYSTEM_PROCESSOR} MATCHES "aarch64")
    message(STATUS "ARM detected")
else()
    message(STATUS "x86 detected")
    if (MSVC)
        set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /arch:AVX")
        set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /arch:AVX")
        set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} /arch:AVX")
    else()
        if(NOT LLAMA_NO_AVX)
            set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -mavx")
        endif()
        if(NOT LLAMA_NO_AVX2)
            set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -mavx2")
        endif()
        if(NOT LLAMA_NO_FMA)
            set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -mfma")
        endif()
        set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -mf16c")
    endif()
endif()

# if (LLAMA_PERF)
#     set(LLAMA_EXTRA_FLAGS ${LLAMA_EXTRA_FLAGS} -DGGML_PERF)
# endif()

add_executable(chat
    chat.cpp
    utils.cpp
    utils.h)

add_executable(quantize
    quantize.cpp
    utils.cpp
    utils.h)

add_library(ggml
    ggml.c
    ggml.h)

target_compile_definitions(ggml PUBLIC ${LLAMA_EXTRA_FLAGS})
target_compile_definitions(chat PUBLIC ${LLAMA_EXTRA_FLAGS})
target_compile_definitions(quantize PUBLIC ${LLAMA_EXTRA_FLAGS})

target_link_libraries(ggml PRIVATE ${LLAMA_EXTRA_LIBS})
target_include_directories(ggml PUBLIC .)
target_link_libraries(quantize PRIVATE ggml)
target_link_libraries(chat PRIVATE ggml)

[anzz1]

The use of F16C code even when F16C was not enabled has been fixed as of a6bdc47

When building for Windows and building with MSVC, you can either uncheck the "LLAMA_AVX2" checkbox in CMake GUI or from command line build it like this:

mkdir build
cd build
cmake -DLLAMA_AVX2=OFF ..
cmake --build . --config Release

However the CMake script currently always declares -mf16c option on non Windows-MSVC builds, so right now you should build with make instead for all other platforms.

make will automatically check your processor features so you don't need to explicitly disable AVX2, it knows you don't have it.

[alextomskras]

Thank you for your help. I'm rebuild and disable avx2 for my cpu. Llama working normal) but slowly))

…

On Tue, Mar 28, 2023, 17:33 anzz1 ***@***.***> wrote: The use of F16C code even when F16C was not enabled has been fixed as of a6bdc47 <a6bdc47> When building for Windows and building with MSVC, you can either uncheck the "LLAMA_AVX2" checkbox in CMake GUI or from command line build it like this: mkdir build cd build cmake -DLLAMA_AVX2=OFF .. cmake --build . --config Release However the CMake script currently always declares -mf16c option on non Windows-MSVC builds, so right now you should build with make instead for all other platforms. make will automatically check your processor features so you don't need to explicitly disable AVX2, it knows you don't have it. — Reply to this email directly, view it on GitHub <#451 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGMC73NS5XSG4YJ4EIGIYMTW6LZB5ANCNFSM6AAAAAAWF7OBP4> . You are receiving this because you commented.Message ID: <ggerganov/llama. ***@***.***>

[perserk]

Hello. I've just added AVX support to a couple of functions. This has significantly increased the speed on my old CPU. #617

[FlowDownTheRiver]

Thank you very much. I haven't tested it yet but I respect the work.

[FlowDownTheRiver]

@perserk > Hello. I've just added AVX support to a couple of functions. This has significantly increased the speed on my old CPU. #617

Is is also possible for https://github.com/ggerganov/whisper.cpp?

[perserk]

@FlowDownTheRiver I see that these changes have already been added with this commit. ggml-org/whisper.cpp@69b8503

[FlowDownTheRiver]

@perserk ahh,I didn't check the repo itself just because of 1 reason. I have tried it with "Subedit" which is used for subtitle editting and they recently added whisper.ccp but as a compiled exe. I tried the UI and exe as standalone saw that my cpu didn't work that is why I thought it was missing. I will try the original repo and ask the subedit developers for an update if that is the case. Thank you very much for letting me know and the work you have done.