Code

metal : optimize FA kernels#10171

Code

Merged

ggerganov merged 9 commits intomasterggerganov/llama.cpp:masterfrom gg/metal-fa-f16ggerganov/llama.cpp:gg/metal-fa-f16Copy head branch name to clipboardNov 8, 2024

+504-345

Commits

Commits on Nov 8, 2024

• ggml : add ggml_flash_attn_ext_get_prec

  ggerganov

  committed
  25e8773

  Copy full SHA for 25e8773

• metal : use F16 precision in FA kernels

  Show description for 7facc29
  ggerganov

  committed
  7facc29

  Copy full SHA for 7facc29

• metal : minor clean-up

  ggerganov

  committed
  2fccc8a

  Copy full SHA for 2fccc8a

• metal : compile-guard bf16 FA kernels

  Show description for 120d512
  ggerganov

  committed
  120d512

  Copy full SHA for 120d512

• build : remove obsolete compile flag [no ci]

  ggerganov

  committed
  486a5eb

  Copy full SHA for 486a5eb

• metal : prevent int overflows [no ci]

  ggerganov

  committed
  5d1a10d

  Copy full SHA for 5d1a10d

• cuda : disable BF16 FA

  Show description for bc143ec
  ggerganov

  committed
  bc143ec

  Copy full SHA for bc143ec

• metal : fix BF16 requirement for FA kernels

  Show description for b89e71b
  ggerganov

  committed
  b89e71b

  Copy full SHA for b89e71b

• make : clean-up [no ci]

  ggerganov

  committed
  a2385da

  Copy full SHA for a2385da