Code

[pull] master from ggerganov:master#158

Code

Closed

pull[bot] wants to merge 73 commits intoteleprint-me:masterteleprint-me/llama.cpp:masterfrom ggerganov:masterggerganov/llama.cpp:masterCopy head branch name to clipboard

+13,233-32,389

Commits

Commits on Nov 30, 2024

• ggml-cpu: replace AArch64 NEON assembly with intrinsics in ggml_gemv_q4_0_4x4_q8_0() (#10567)

  Show description for 0c39f44
  angt

  authored
  0c39f44

  Copy full SHA for 0c39f44

Commits on Dec 1, 2024

• build: update Makefile comments for C++ version change (#10598)

  wangqin0

  authored
  43957ef

  Copy full SHA for 43957ef

• readme : update the usage section with examples (#10596)

  Show description for 6acce39
  ggerganov

  authored
  6acce39

  Copy full SHA for 6acce39

• server : bind to any port when specified (#10590)

  alek3y

  authored
  86dc11c

  Copy full SHA for 86dc11c

• ggml : automatic selection of best CPU backend (#10606)

  Show description for 3420909
  slaren

  authored
  3420909

  Copy full SHA for 3420909

• ci: add error handling for Python venv creation in run.sh (#10608)

  wangqin0

  authored
  5c7a5aa

  Copy full SHA for 5c7a5aa

• grammars : add English-only grammar (#10612)

  ggerganov

  authored
  5e1ed95

  Copy full SHA for 5e1ed95

• Add mistral-v1, mistral-v3, mistral-v3-tekken and mistral-v7 chat template types (#10572)

  Show description for 917786f
  jukofyork

  authored
  917786f

  Copy full SHA for 917786f

Commits on Dec 2, 2024

• contrib : refresh (#10593)

  Show description for 4cb003d
  ggerganov

  authored
  4cb003d

  Copy full SHA for 4cb003d

• SYCL: Fix and switch to GGML_LOG system instead of fprintf (#10579)

  Show description for 991f8aa
  qnixsynapse

  authored
  991f8aa

  Copy full SHA for 991f8aa

• server: Add "tokens per second" information in the backend (#10548)

  Show description for 64ed209
  

  lhpqaq

  and

  ngxson

  authored
  64ed209

  Copy full SHA for 64ed209

• make : deprecate (#10514)

  Show description for 8648c52
  

  ggerganov

  and

  slaren

  authored
  8648c52

  Copy full SHA for 8648c52

• llama : add enum for built-in chat templates (#10623)

  Show description for 642330a
  ngxson

  authored
  642330a

  Copy full SHA for 642330a

Commits on Dec 3, 2024

• server : fix default draft model parameters (#10586)

  Show description for 70b98fa
  ggerganov

  authored
  70b98fa

  Copy full SHA for 70b98fa

• github : minify link [no ci]

  ggerganov

  committed
  844e2e1

  Copy full SHA for 844e2e1

• github : minify link [no ci] (revert)

  Show description for 515d4e5
  ggerganov

  committed
  515d4e5

  Copy full SHA for 515d4e5

• metal : small-batch mat-mul kernels (#10581)

  Show description for 0115df2
  ggerganov

  authored
  0115df2

  Copy full SHA for 0115df2

• readme : add option, update default value, fix formatting (#10271)

  Show description for 82bca22
  pothitos

  authored
  82bca22

  Copy full SHA for 82bca22

• llama : add missing LLAMA_API for llama_chat_builtin_templates (#10636)

  ngxson

  authored
  3b4f2e3

  Copy full SHA for 3b4f2e3

• metal : add GGML_OP_CONV_TRANSPOSE_1D kernels (ggml/1026)

  Show description for 667d70d
  

  PABannier

  authored and

  ggerganov

  committed
  667d70d

  Copy full SHA for 667d70d

• feat: add GGML_UNARY_OP_ARGMAX Metal kernel (ggml/1019)

  Show description for efb6ae9
  

  2 peopleauthored and

  ggerganov

  committed
  efb6ae9

  Copy full SHA for efb6ae9

• CUDA: remove unnecessary warp reduce in FA (ggml/1032)

  Show description for e9e661b
  

  2 peopleauthored and

  ggerganov

  committed
  e9e661b

  Copy full SHA for e9e661b

• sync : ggml

  ggerganov

  committed
  c505471

  Copy full SHA for c505471

• scripts : remove amx sync

  Show description for 1cd3df4
  ggerganov

  committed
  1cd3df4

  Copy full SHA for 1cd3df4

• server : (web ui) Various improvements, now use vite as bundler (#10599)

  Show description for 91c36c2
  ngxson

  authored
  91c36c2

  Copy full SHA for 91c36c2

• vulkan: optimize and reenable split_k (#10637)

  Show description for cc98896
  jeffbolznv

  authored
  cc98896

  Copy full SHA for cc98896

Commits on Dec 4, 2024

• clip : add sycl support (#10574)

  Show description for 01e6d9b
  

  piDack

  and

  piDack

  authored
  01e6d9b

  Copy full SHA for 01e6d9b

• Add docs for creating a static build (#10268) (#10630)

  Show description for da6aac9
  

  mostlygeek

  and

  slaren

  authored
  da6aac9

  Copy full SHA for da6aac9

• Avoid using __fp16 on ARM with old nvcc (#10616)

  frankier

  authored
  cd2f37b

  Copy full SHA for cd2f37b

• fix typo of README.md (#10605)

  WrRan

  authored
  98036d5

  Copy full SHA for 98036d5

• SYCL : Move to compile time oneMKL interface backend selection for NVIDIA backend (#10584)

  Show description for 40c6d79
  s-Nick

  authored
  40c6d79

  Copy full SHA for 40c6d79

• vulkan: Implement "fast divide" (mul+shift) for unary ops like copy (#10642)

  jeffbolznv

  authored
  2759916

  Copy full SHA for 2759916

• llama: Support MiniCPM-1B (with & w/o longrope) (#10559)

  JFLFY2255

  authored
  8d0cfd5

  Copy full SHA for 8d0cfd5

• Fix HF repo commit to clone lora test models (#10649)

  ltoniazzi

  authored
  253b7fd

  Copy full SHA for 253b7fd

• ggml-cpu : fix HWCAP2_I8MM value (#10646)

  slaren

  authored
  2803540

  Copy full SHA for 2803540

• ggml : add predefined list of CPU backend variants to build (#10626)

  Show description for 59f4db1
  slaren

  authored
  59f4db1

  Copy full SHA for 59f4db1

• server : fix speculative decoding with context shift (#10641)

  Show description for 1da7b76
  ggerganov

  authored
  1da7b76

  Copy full SHA for 1da7b76

• Update deprecation-warning.cpp (#10619)

  Show description for f112d19
  aryantandon01

  authored
  f112d19

  Copy full SHA for f112d19

Commits on Dec 5, 2024

• py : update outdated copy-paste instructions [no ci] (#10667)

  Show description for d405804
  danbev

  authored
  d405804

  Copy full SHA for d405804

• ggml : add GGML_PAD_REFLECT_1D operation (ggml/1034)

  Show description for c2082d9
  

  PABannier

  authored and

  ggerganov

  committed
  c2082d9

  Copy full SHA for c2082d9

• ggml: add GGML_SET Metal kernel + i32 CPU kernel (ggml/1037)

  Show description for a8cbab2
  

  PABannier

  authored and

  ggerganov

  committed
  a8cbab2

  Copy full SHA for a8cbab2

• sync : ggml

  ggerganov

  committed
  0cd182e

  Copy full SHA for 0cd182e

• llama : add Minerva 7B model support (#10673)

  Show description for 6fe6247
  Riccorl

  authored
  6fe6247

  Copy full SHA for 6fe6247

• vulkan: Add VK_NV_cooperative_matrix2 support for mul_mat and flash attention (#10206)

  jeffbolznv

  authored
  c9c6e01

  Copy full SHA for c9c6e01

• fix(server) : not show alert when DONE is received (#10674)

  pminev

  authored
  7736837

  Copy full SHA for 7736837

Commits on Dec 6, 2024

• server : (refactoring) do not rely on JSON internally (#10643)

  Show description for 6c5bc06
  ngxson

  authored
  6c5bc06

  Copy full SHA for 6c5bc06

• common : bring back --no-warmup to server (#10686)

  ngxson

  authored
  f162d45

  Copy full SHA for f162d45

• convert : add custom attention mapping

  ggerganov

  committed
  c5ede38

  Copy full SHA for c5ede38

Commits on Dec 7, 2024

• convert : add support for Roberta embeddings (#10695)

  Ssukriti

  authored
  784a14a

  Copy full SHA for 784a14a

• metal : Extend how Llama.cpp locates metal resources (#10676)

  Show description for 86a1934
  

  ormandi

  and

  ggerganov

  authored
  86a1934

  Copy full SHA for 86a1934

• Vulkan: VK_KHR_cooperative_matrix support to speed up prompt processing (#10597)

  Show description for 3df784b
  0cc4m

  authored
  3df784b

  Copy full SHA for 3df784b

• server : fix free of spec context and batch (#10651)

  Show description for c2a16c0
  ggerganov

  authored
  c2a16c0

  Copy full SHA for c2a16c0

• ggml : refactor online repacking (#10446)

  Show description for 19d8762
  

  Djip007

  and

  ggerganov

  authored
  19d8762

  Copy full SHA for 19d8762

• server : various fixes (#10704)

  Show description for ce4a7b8
  

  ggerganov

  and

  ngxson

  authored
  ce4a7b8

  Copy full SHA for ce4a7b8

• ggml : disable iq4_nl interleave size 8 (#10709)

  Show description for d9c3ba2
  ggerganov

  authored
  d9c3ba2

  Copy full SHA for d9c3ba2

• server : (refactor) no more json in server_task input (#10691)

  Show description for 3573fa8
  ngxson

  authored
  3573fa8

  Copy full SHA for 3573fa8

• llama : add 128k yarn context for Qwen (#10698)

  Show description for 62e84d9
  robbiemu

  authored
  62e84d9

  Copy full SHA for 62e84d9

Commits on Dec 8, 2024

• vulkan: compile a test shader in cmake to check for coopmat2 support (#10713)

  jeffbolznv

  authored
  ecc93d0

  Copy full SHA for ecc93d0

• llama : use cmake for swift build (#10525)

  Show description for 43ed389
  

  slaren

  and

  ggerganov

  authored
  43ed389

  Copy full SHA for 43ed389

• Vulkan: fix NaN in tanh.comp with AMD proprietary driver on Windows (#10723)

  Show description for 06d7014
  stduhpf

  authored
  06d7014

  Copy full SHA for 06d7014

• server : bring back info of final chunk in stream mode (#10722)

  Show description for e52522b
  ngxson

  authored
  e52522b

  Copy full SHA for e52522b

• server : fix format_infill (#10724)

  Show description for ce8784b
  ngxson

  authored
  ce8784b

  Copy full SHA for ce8784b

Commits on Dec 9, 2024

• cmake : simplify msvc charsets (#10672)

  iboB

  authored
  1a05004

  Copy full SHA for 1a05004

• vulkan: fix compile warnings (#10731)

  jeffbolznv

  authored
  3d98b4c

  Copy full SHA for 3d98b4c

• Changes to CMakePresets.json to add ninja clang target on windows (#10668)

  Show description for c37fb4c
  

  Srihari-mcw

  and

  max-krasnyansky

  authored
  c37fb4c

  Copy full SHA for c37fb4c

• CUDA: fix shared memory access condition for mmv (#10740)

  JohannesGaessler

  authored
  26a8406

  Copy full SHA for 26a8406

Commits on Dec 10, 2024

• vulkan: disable spirv-opt for coopmat shaders (#10763)

  Show description for a05e2af
  jeffbolznv

  authored
  a05e2af

  Copy full SHA for a05e2af

• server : add flag to disable the web-ui (#10762) (#10751)

  Show description for a86ad84
  

  eugeniosegala

  and

  eugenio.segala

  authored
  a86ad84

  Copy full SHA for a86ad84

• CUDA: rename macros to avoid conflicts with WinAPI (#10736)

  Show description for 750cb3e
  aendk

  authored
  750cb3e

  Copy full SHA for 750cb3e

• imatrix : Add imatrix to --no-context-shift (#10766)

  Show description for ae4b922
  bartowski1182

  authored
  ae4b922

  Copy full SHA for ae4b922

• vulkan: dynamic subgroup size for the remaining k quants (#10745)

  Show description for dafae66
  netrunnereve

  authored
  dafae66

  Copy full SHA for dafae66

• vulkan: request round-to-even for fp16 in im2col/rope_head (#10767)

  Show description for b685daf
  jeffbolznv

  authored
  b685daf

  Copy full SHA for b685daf

Commits on Dec 11, 2024

• ggml: load all backends from a user-provided search path (#10699)

  Show description for 43041d2
  

  giladgd

  and

  slaren

  authored
  43041d2

  Copy full SHA for 43041d2