Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build : detect AVX512 in Makefile, add AVX512 option in CMake #2043

Merged
merged 8 commits into from
Apr 15, 2024
Merged

build : detect AVX512 in Makefile, add AVX512 option in CMake #2043

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
c3753f9
make : add AVX512 detection to Makefile and CMakeLists.txt
didzis Apr 12, 2024
e8ee795
make : autodetect more AVX512 instruction subsets
didzis Apr 12, 2024
eb24a77
cmake : do not default to AVX512, must be enabled explicitly
didzis Apr 12, 2024
ef25a4a
cmake : enable a set of AVX512 subsets, when AVX512 is turned on
didzis Apr 12, 2024
dc69460
make : consolidate AVX512 subsets, add AVX512 VBMI
didzis Apr 12, 2024
f7c4acb
cmake : revert to NO AVX512 setting, add settings for AVX512 VNNI and…
didzis Apr 12, 2024
83c902b
make : re-introduce AVX512VNNI back
didzis Apr 12, 2024
9b0b1d1
cmake : remove superfluous comment line
didzis Apr 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 37 additions & 11 deletions CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,10 +59,13 @@ option(WHISPER_BUILD_EXAMPLES "whisper: build examples" ${WHISPER_STANDA

option(WHISPER_SDL2 "whisper: support for libSDL2" OFF)

option(WHISPER_NO_AVX "whisper: disable AVX" OFF)
option(WHISPER_NO_AVX2 "whisper: disable AVX2" OFF)
option(WHISPER_NO_FMA "whisper: disable FMA" OFF)
option(WHISPER_NO_F16C "whisper: disable F16c" OFF)
option(WHISPER_NO_AVX "whisper: disable AVX" OFF)
option(WHISPER_NO_AVX2 "whisper: disable AVX2" OFF)
option(WHISPER_NO_AVX512 "whisper: disable AVX512" ON)
option(WHISPER_NO_AVX512_VBMI "whisper: disable AVX512-VBMI" ON)
option(WHISPER_NO_AVX512_VNNI "whisper: disable AVX512-VNNI" ON)
option(WHISPER_NO_FMA "whisper: disable FMA" OFF)
option(WHISPER_NO_F16C "whisper: disable F16c" OFF)

option(WHISPER_OPENVINO "whisper: support for OpenVINO" OFF)

Expand Down Expand Up @@ -464,16 +467,30 @@ else()
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /utf-8")
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /utf-8")
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} /utf-8")
if(NOT WHISPER_NO_AVX2)
if(NOT WHISPER_NO_AVX512)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /arch:AVX512")
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /arch:AVX512")
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} /arch:AVX512")
# MSVC has no compile-time flags enabling specific
# AVX512 extensions, neither it defines the
# macros corresponding to the extensions.
# Do it manually.
if (NOT WHISPER_NO_AVX512_VBMI)
add_compile_definitions($<$<COMPILE_LANGUAGE:C>:__AVX512VBMI__>)
add_compile_definitions($<$<COMPILE_LANGUAGE:CXX>:__AVX512VBMI__>)
endif()
if (NOT WHISPER_NO_AVX512_VNNI)
add_compile_definitions($<$<COMPILE_LANGUAGE:C>:__AVX512VNNI__>)
add_compile_definitions($<$<COMPILE_LANGUAGE:CXX>:__AVX512VNNI__>)
endif()
elseif(NOT WHISPER_NO_AVX2)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /arch:AVX2")
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /arch:AVX2")
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} /arch:AVX2")
else()
if(NOT WHISPER_NO_AVX)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /arch:AVX")
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /arch:AVX")
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} /arch:AVX")
endif()
elseif(NOT WHISPER_NO_AVX)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /arch:AVX")
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /arch:AVX")
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} /arch:AVX")
endif()
else()
if (EMSCRIPTEN)
Expand All @@ -486,6 +503,15 @@ else()
if(NOT WHISPER_NO_AVX2)
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -mavx2")
endif()
if(NOT WHISPER_NO_AVX512)
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -mavx512f -mavx512cd -mavx512vl -mavx512dq -mavx512bw")
endif()
if(NOT WHISPER_NO_AVX512_VBMI)
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -mavx512vbmi")
endif()
if(NOT WHISPER_NO_AVX512_VNNI)
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -mavx512vnni")
endif()
if(NOT WHISPER_NO_FMA)
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -mfma")
endif()
Expand Down
18 changes: 18 additions & 0 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,24 @@ ifeq ($(UNAME_M),$(filter $(UNAME_M),x86_64 i686 amd64))
CXXFLAGS += -mavx2
endif

AVX512F_M := $(shell $(CPUINFO_CMD) | grep -iw 'AVX512F')
ifneq (,$(AVX512F_M))
CFLAGS += -mavx512f -mavx512cd -mavx512vl -mavx512dq -mavx512bw
CXXFLAGS += -mavx512f -mavx512cd -mavx512vl -mavx512dq -mavx512bw
endif

AVX512VNNI_M := $(shell $(CPUINFO_CMD) | grep -iwE 'AVX512_VNNI|AVX512VNNI')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AVX512_VNNI is for Linux, so just out of curiosity:
Which OS uses AVX512VNNI?

As you're adding VNNI beside basic AVX-512 subsets, I would suggest adding also VBMI, as llama.cpp does it too. It's nice to have some kind of parity between these projects.

Moreover, if you're adding VNNI and VBMI in Makefile, you should add them in CMakeLists.txt too, via WHISPER_NO_AVX512_VNNI and WHISPER_NO_AVX512_VBMI, defaulting to ON. I suggest using add_compile_definitions similarly like llama.cpp does.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added AVX512VNNI for pure symmetry just in case any system does have it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I asked about AVX512VNNI, didn't request removing. But I see you removed it in dc69460.

So I looked around and it seems AVX512VNNI is the name reported by FreeBSD.
http://fxr.watson.org/fxr/source/x86/x86/identcpu.c#L994

Please bring back grep -iwE 'AVX512_VNNI|AVX512VNNI' for VNNI detection.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, my feeling was right then.

ifneq (,$(AVX512VNNI_M))
CFLAGS += -mavx512vnni
CXXFLAGS += -mavx512vnni
endif

AVX512VBMI_M := $(shell $(CPUINFO_CMD) | grep -iw 'AVX512VBMI')
ifneq (,$(AVX512VBMI_M))
CFLAGS += -mavx512vbmi
CXXFLAGS += -mavx512vbmi
endif

FMA_M := $(shell $(CPUINFO_CMD) | grep -iw 'FMA')
ifneq (,$(FMA_M))
CFLAGS += -mfma
Expand Down
Loading