Code

1#30

Code

Merged

apicalshark merged 96 commits intomasterapicalshark/llama.cpp:masterfrom 1apicalshark/llama.cpp:1Copy head branch name to clipboardDec 19, 2024

+24,709-5,437

Commits

Commits on Dec 5, 2024

• ggml : add GGML_PAD_REFLECT_1D operation (ggml/1034)

  Show description for c2082d9
  

  PABannier

  authored and

  ggerganov

  committed
  c2082d9

  Copy full SHA for c2082d9

• ggml: add GGML_SET Metal kernel + i32 CPU kernel (ggml/1037)

  Show description for a8cbab2
  

  PABannier

  authored and

  ggerganov

  committed
  a8cbab2

  Copy full SHA for a8cbab2

• sync : ggml

  ggerganov

  committed
  0cd182e

  Copy full SHA for 0cd182e

• llama : add Minerva 7B model support (ggerganov#10673)

  Show description for 6fe6247
  Riccorl

  authored
  6fe6247

  Copy full SHA for 6fe6247

• vulkan: Add VK_NV_cooperative_matrix2 support for mul_mat and flash attention (ggerganov#10206)

  jeffbolznv

  authored
  c9c6e01

  Copy full SHA for c9c6e01

• fix(server) : not show alert when DONE is received (ggerganov#10674)

  pminev

  authored
  7736837

  Copy full SHA for 7736837

Commits on Dec 6, 2024

• server : (refactoring) do not rely on JSON internally (ggerganov#10643)

  Show description for 6c5bc06
  ngxson

  authored
  6c5bc06

  Copy full SHA for 6c5bc06

• common : bring back --no-warmup to server (ggerganov#10686)

  ngxson

  authored
  f162d45

  Copy full SHA for f162d45

• convert : add custom attention mapping

  ggerganov

  committed
  c5ede38

  Copy full SHA for c5ede38

Commits on Dec 7, 2024

• convert : add support for Roberta embeddings (ggerganov#10695)

  Ssukriti

  authored
  784a14a

  Copy full SHA for 784a14a

• metal : Extend how Llama.cpp locates metal resources (ggerganov#10676)

  Show description for 86a1934
  

  ormandi

  and

  ggerganov

  authored
  86a1934

  Copy full SHA for 86a1934

• Vulkan: VK_KHR_cooperative_matrix support to speed up prompt processing (ggerganov#10597)

  Show description for 3df784b
  0cc4m

  authored
  3df784b

  Copy full SHA for 3df784b

• server : fix free of spec context and batch (ggerganov#10651)

  Show description for c2a16c0
  ggerganov

  authored
  c2a16c0

  Copy full SHA for c2a16c0

• ggml : refactor online repacking (ggerganov#10446)

  Show description for 19d8762
  

  Djip007

  and

  ggerganov

  authored
  19d8762

  Copy full SHA for 19d8762

• server : various fixes (ggerganov#10704)

  Show description for ce4a7b8
  

  ggerganov

  and

  ngxson

  authored
  ce4a7b8

  Copy full SHA for ce4a7b8

• ggml : disable iq4_nl interleave size 8 (ggerganov#10709)

  Show description for d9c3ba2
  ggerganov

  authored
  d9c3ba2

  Copy full SHA for d9c3ba2

• server : (refactor) no more json in server_task input (ggerganov#10691)

  Show description for 3573fa8
  ngxson

  authored
  3573fa8

  Copy full SHA for 3573fa8

• llama : add 128k yarn context for Qwen (ggerganov#10698)

  Show description for 62e84d9
  robbiemu

  authored
  62e84d9

  Copy full SHA for 62e84d9

Commits on Dec 8, 2024

• vulkan: compile a test shader in cmake to check for coopmat2 support (ggerganov#10713)

  jeffbolznv

  authored
  ecc93d0

  Copy full SHA for ecc93d0

• llama : use cmake for swift build (ggerganov#10525)

  Show description for 43ed389
  

  slaren

  and

  ggerganov

  authored
  43ed389

  Copy full SHA for 43ed389

• Vulkan: fix NaN in tanh.comp with AMD proprietary driver on Windows (ggerganov#10723)

  Show description for 06d7014
  stduhpf

  authored
  06d7014

  Copy full SHA for 06d7014

• server : bring back info of final chunk in stream mode (ggerganov#10722)

  Show description for e52522b
  ngxson

  authored
  e52522b

  Copy full SHA for e52522b

• server : fix format_infill (ggerganov#10724)

  Show description for ce8784b
  ngxson

  authored
  ce8784b

  Copy full SHA for ce8784b

Commits on Dec 9, 2024

• cmake : simplify msvc charsets (ggerganov#10672)

  iboB

  authored
  1a05004

  Copy full SHA for 1a05004

• vulkan: fix compile warnings (ggerganov#10731)

  jeffbolznv

  authored
  3d98b4c

  Copy full SHA for 3d98b4c

• Changes to CMakePresets.json to add ninja clang target on windows (ggerganov#10668)

  Show description for c37fb4c
  

  Srihari-mcw

  and

  max-krasnyansky

  authored
  c37fb4c

  Copy full SHA for c37fb4c

• CUDA: fix shared memory access condition for mmv (ggerganov#10740)

  JohannesGaessler

  authored
  26a8406

  Copy full SHA for 26a8406

Commits on Dec 10, 2024

• vulkan: disable spirv-opt for coopmat shaders (ggerganov#10763)

  Show description for a05e2af
  jeffbolznv

  authored
  a05e2af

  Copy full SHA for a05e2af

• server : add flag to disable the web-ui (ggerganov#10762) (ggerganov#10751)

  Show description for a86ad84
  

  eugeniosegala

  and

  eugenio.segala

  authored
  a86ad84

  Copy full SHA for a86ad84

• CUDA: rename macros to avoid conflicts with WinAPI (ggerganov#10736)

  Show description for 750cb3e
  aendk

  authored
  750cb3e

  Copy full SHA for 750cb3e

• imatrix : Add imatrix to --no-context-shift (ggerganov#10766)

  Show description for ae4b922
  bartowski1182

  authored
  ae4b922

  Copy full SHA for ae4b922

• vulkan: dynamic subgroup size for the remaining k quants (ggerganov#10745)

  Show description for dafae66
  netrunnereve

  authored
  dafae66

  Copy full SHA for dafae66

• vulkan: request round-to-even for fp16 in im2col/rope_head (ggerganov#10767)

  Show description for b685daf
  jeffbolznv

  authored
  b685daf

  Copy full SHA for b685daf

Commits on Dec 11, 2024

• ggml: load all backends from a user-provided search path (ggerganov#10699)

  Show description for 43041d2
  

  giladgd

  and

  slaren

  authored
  43041d2

  Copy full SHA for 43041d2

• docs: fix server documentation formatting (ggerganov#10776)

  CentricStorm

  authored
  4b4d92b

  Copy full SHA for 4b4d92b

• bug-fix: snprintf prints NULL in place of the last character (ggerganov#10419)

  Show description for 484d2f3
  kallewoof

  authored
  484d2f3

  Copy full SHA for 484d2f3

• ci : pin nodejs to 22.11.0 (ggerganov#10779)

  ngxson

  authored
  92f77a6

  Copy full SHA for 92f77a6

• Update README.md (ggerganov#10772)

  qingy1337

  authored
  1a31d0d

  Copy full SHA for 1a31d0d

• server : (UI) add tok/s, get rid of completion.js (ggerganov#10786)

  Show description for 235f6e1
  ngxson

  authored
  235f6e1

  Copy full SHA for 235f6e1

• gguf-py : bump version to 0.11.0

  ggerganov

  committed
  fb18934

  Copy full SHA for fb18934

• Merge pull request ggerganov#10788 from ggerganov/gg/gguf-py-0.11.0

  ggerganov

  authored
  973f328

  Copy full SHA for 973f328

• docs: update server streaming mode documentation (ggerganov#9519)

  Show description for 5555c0c
  CentricStorm

  authored
  5555c0c

  Copy full SHA for 5555c0c

Commits on Dec 12, 2024

• common : add missing env var for speculative (ggerganov#10801)

  ngxson

  authored
  9fdb124

  Copy full SHA for 9fdb124

• Vulkan: Add VK_EXT_subgroup_size_control support to ensure full subgroups for coopmats (ggerganov#10721)

  Show description for dc5301d
  0cc4m

  authored
  dc5301d

  Copy full SHA for dc5301d

• Vulkan: Use improved q4_k and q5_k dequant code in dequant shaders (ggerganov#10798)

  0cc4m

  authored
  4064c0e

  Copy full SHA for 4064c0e

• remove CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS (ggerganov#10797)

  Show description for cb13ef8
  slaren

  authored
  cb13ef8

  Copy full SHA for cb13ef8

• CUDA: faster non-contiguous concat (ggerganov#10760)

  Show description for 8faa1d4
  

  A3shTnT

  and

  slaren

  authored
  8faa1d4

  Copy full SHA for 8faa1d4

• contrib : add ngxson as codeowner (ggerganov#10804)

  ngxson

  authored
  274ec65

  Copy full SHA for 274ec65

• common : improve -ctv -ctk CLI arguments (ggerganov#10806)

  Show description for adffa6f
  ngxson

  authored
  adffa6f

  Copy full SHA for adffa6f

Commits on Dec 13, 2024

• ggml : Fix compilation issues on ARM platform when building without fp16 (ggerganov#10811)

  kkontny

  authored
  d583cd0

  Copy full SHA for d583cd0

• SYCL: Reduce most of the compiler warnings (ggerganov#10748)

  Show description for 83ed24a
  

  qnixsynapse

  and

  abhilash1910

  authored
  83ed24a

  Copy full SHA for 83ed24a

• vulkan: small mul_mat_vec optimizations (ggerganov#10665)

  Show description for 64ae065
  netrunnereve

  authored
  64ae065

  Copy full SHA for 64ae065

• Fix crash caused by ggml_backend_load_all when launching on Android Activity (ggerganov#10812)

  Show description for 9f35e44
  

  sienaiwun

  and

  slaren

  authored
  9f35e44

  Copy full SHA for 9f35e44

• gguf-py : numpy 2 newbyteorder fix (ggerganov#9772)

  jettjaniak

  authored
  4601a8b

  Copy full SHA for 4601a8b

• fix: graceful shutdown for Docker images (ggerganov#10815)

  co42

  authored
  11e07fd

  Copy full SHA for 11e07fd

• Opt class for positional argument handling (ggerganov#10508)

  Show description for c27ac67
  ericcurtin

  authored
  c27ac67

  Copy full SHA for c27ac67

• Introducing experimental OpenCL backend with support for Qualcomm Adreno GPUs (ggerganov#10693)

  Show description for a76c56f
  

  6 peopleauthored
  a76c56f

  Copy full SHA for a76c56f

• Removes spurious \r in output that causes logging in journalctl to treat lines as binary and therefore hidden by default (ggerganov#10771)

  Show description for 56eea07
  

  cduk

  and

  Charles Darke

  authored
  56eea07

  Copy full SHA for 56eea07

Commits on Dec 14, 2024

• llama : add Qwen2VL support + multimodal RoPE (ggerganov#10361)

  Show description for ba1cb19
  

  HimariO

  and

  ggerganov

  authored
  ba1cb19

  Copy full SHA for ba1cb19

• nix: allow to override rocm gpu targets (ggerganov#10794)

  Show description for e52aba5
  kurnevsky

  authored
  e52aba5

  Copy full SHA for e52aba5

• server: Fix has_next_line in JSON response (ggerganov#10818)

  Show description for 89d604f
  MichelleTanPY

  authored
  89d604f

  Copy full SHA for 89d604f

Commits on Dec 15, 2024

• gguf-py : bump to v0.13.0

  ggerganov

  committed
  b5ae1dd

  Copy full SHA for b5ae1dd

• server: (UI) add syntax highlighting and latex math rendering (ggerganov#10808)

  Show description for 5478bbc
  

  VJHack

  and

  ngxson

  authored
  5478bbc

  Copy full SHA for 5478bbc

• scripts : change build path to "build-bench" for compare-commits.sh (ggerganov#10836)

  ggerganov

  authored
  87cf323

  Copy full SHA for 87cf323

• llama : add Deepseek MoE v1 & GigaChat models (ggerganov#10827)

  Show description for a097415
  Inf1delis

  authored
  a097415

  Copy full SHA for a097415

• llava : Allow locally downloaded models for QwenVL (ggerganov#10833)

  Show description for 4ddd199
  

  bartowski1182

  and

  ngxson

  authored
  4ddd199

  Copy full SHA for 4ddd199

Commits on Dec 16, 2024

• sampling : refactor + optimize penalties sampler (ggerganov#10803)

  Show description for 644fd71
  

  ggerganov

  and

  slaren

  authored
  644fd71

  Copy full SHA for 644fd71

• unicode : improve naming style (ggerganov#10838)

  Show description for 08ea539
  ggerganov

  authored
  08ea539

  Copy full SHA for 08ea539

• rwkv6: add wkv6 support for Vulkan backend (ggerganov#10829)

  Show description for 160bc03
  

  uniartisan

  and

  MollySophia

  authored
  160bc03

  Copy full SHA for 160bc03

Commits on Dec 17, 2024

• vulkan: bugfixes for small subgroup size systems + llvmpipe test (ggerganov#10809)

  Show description for 7b1ec53
  netrunnereve

  authored
  7b1ec53

  Copy full SHA for 7b1ec53

• server : (UI) fix missing async generator on safari (ggerganov#10857)

  Show description for 227d7c5
  ngxson

  authored
  227d7c5

  Copy full SHA for 227d7c5

• readme : update typos (ggerganov#10863)

  ruanych

  authored
  4f51968

  Copy full SHA for 4f51968

• llama : add Falcon3 support (ggerganov#10864)

  mokeddembillel

  authored
  382bc7f

  Copy full SHA for 382bc7f

• server : fill usage info in embeddings and rerank responses (ggerganov#10852)

  Show description for 05c3a44
  krystiancha

  authored
  05c3a44

  Copy full SHA for 05c3a44

• ggml : update ggml_backend_cpu_device_supports_op (ggerganov#10867)

  Show description for 0006f5a
  ggerganov

  authored
  0006f5a

  Copy full SHA for 0006f5a

• ggml : add check for grad_accs (ggml/1046)

  Show description for 3919da8
  

  danbev

  authored and

  ggerganov

  committed
  3919da8

  Copy full SHA for 3919da8

• ggml : remove return from ggml_gallocr_allocate_node (ggml/1048)

  Show description for 130d0c9
  

  danbev

  authored and

  ggerganov

  committed
  130d0c9

  Copy full SHA for 130d0c9

• vulkan : fix soft_max.comp division by zero (whisper/2633)

  Show description for 8dd19a4
  

  gn64

  authored and

  ggerganov

  committed
  8dd19a4

  Copy full SHA for 8dd19a4

• cmake : fix "amd64" processor string (whisper/2638)

  ggerganov

  committed
  78f7667

  Copy full SHA for 78f7667

• sync : ggml

  ggerganov

  committed
  5437d4a

  Copy full SHA for 5437d4a

• tests: add tests for GGUF (ggerganov#10830)

  JohannesGaessler

  authored
  081b29b

  Copy full SHA for 081b29b

• Use model->gguf_kv for loading the template instead of using the C API. (ggerganov#10868)

  Show description for d62b532
  dranger003

  authored
  d62b532

  Copy full SHA for d62b532

Commits on Dec 18, 2024

• Revert "llama : add Falcon3 support (ggerganov#10864)" (ggerganov#10876)

  Show description for 4da69d1
  slaren

  authored
  4da69d1

  Copy full SHA for 4da69d1

• docs: Fix HIP (née hipBLAS) in README (ggerganov#10880)

  Show description for 6b064c9
  brianredbeard

  authored
  6b064c9

  Copy full SHA for 6b064c9

• server : (embeddings) using same format for "input" and "content" (ggerganov#10872)

  Show description for 4682887
  ngxson

  authored
  4682887

  Copy full SHA for 4682887

• server : add "tokens" output (ggerganov#10853)

  Show description for 0e70ba6
  

  ggerganov

  and

  ngxson

  authored
  0e70ba6

  Copy full SHA for 0e70ba6

• server : output embeddings for all tokens when pooling = none (ggerganov#10861)

  Show description for 152610e
  

  ggerganov

  and

  ngxson

  authored
  152610e

  Copy full SHA for 152610e

• server: avoid overwriting Authorization header (ggerganov#10878)

  Show description for 7bbb5ac
  

  vesath

  and

  ngxson

  authored
  7bbb5ac

  Copy full SHA for 7bbb5ac

• tts : add OuteTTS support (ggerganov#10784)

  Show description for 0bf2d10
  ggerganov

  authored
  0bf2d10

  Copy full SHA for 0bf2d10

• ggml : fix arm build (ggerganov#10890)

  Show description for 9177484
  

  slaren

  and

  angt

  authored
  9177484

  Copy full SHA for 9177484

Commits on Dec 19, 2024

• llama-run : improve progress bar (ggerganov#10821)

  Show description for 7909e85
  ericcurtin

  authored
  7909e85

  Copy full SHA for 7909e85

• tests: disable GGUF test for bad value size (ggerganov#10886)

  JohannesGaessler

  authored
  cd920d0

  Copy full SHA for cd920d0

• convert : Add support for Microsoft Phi-4 model (ggerganov#10817)

  Show description for 7585edb
  

  fairydreaming

  and

  sszymczy

  authored
  7585edb

  Copy full SHA for 7585edb

• llama : fix Roberta embeddings (ggerganov#10856)

  Show description for 2fffc52
  

  Ssukriti

  and

  gabe-l-hart

  authored
  2fffc52

  Copy full SHA for 2fffc52

• ggml: fix arm build with gcc (ggerganov#10895)

  Show description for a3c33b1
  angt

  authored
  a3c33b1

  Copy full SHA for a3c33b1

• Merge branch 'master' into 1

  apicalshark

  authored
  4ec10e6

  Copy full SHA for 4ec10e6