Code

merge from upstream#22

Code

Merged

l3utterfly merged 84 commits intolayla-buildl3utterfly/llama.cpp:layla-buildfrom mergel3utterfly/llama.cpp:mergeCopy head branch name to clipboardJun 5, 2024

+112,050-47,851

Commits

Commits on May 27, 2024

• metal : add GGML_OP_REPEAT kernels (ggerganov#7557)

  Show description for 1d8fca7
  ggerganov

  authored
  1d8fca7

  Copy full SHA for 1d8fca7

• Add freq factors (ggerganov#7495)

  AidanBeltonS

  authored
  5487593

  Copy full SHA for 5487593

• Fix q_xxs using mul_mat_q (ggerganov#7459)

  AidanBeltonS

  authored
  95f84d5

  Copy full SHA for 95f84d5

• Allow multiple copy function pointers for CUDA graph kernel param updates (ggerganov#7565)

  Show description for 197c006
  agray3

  authored
  197c006

  Copy full SHA for 197c006

• make: add --device-debug to NVCC debug flags (ggerganov#7542)

  JohannesGaessler

  authored
  10b1e45

  Copy full SHA for 10b1e45

• adding in x64 targets to cmake presets (ggerganov#7574)

  kunnis

  authored
  0136966

  Copy full SHA for 0136966

• update HIP_UMA ggerganov#7399 (ggerganov#7414)

  Show description for 852aafb
  

  Djip007

  and

  slaren

  authored
  852aafb

  Copy full SHA for 852aafb

Commits on May 28, 2024

• llava : update clip.h (ggerganov#7580)

  Show description for 74b239b
  eltociear

  authored
  74b239b

  Copy full SHA for 74b239b

• Markdownish code block fix (ggerganov#7571)

  Show description for c417671
  nathan-sixnines

  authored
  c417671

  Copy full SHA for c417671

• server: do not remove whitespace at the start of a completion chunk (ggerganov#7524)

  mgroeber9110

  authored
  9335b96

  Copy full SHA for 9335b96

• ggml : generalize GGML_OP_CONCAT (ggerganov#7563)

  Show description for 0548a41
  ggerganov

  authored
  0548a41

  Copy full SHA for 0548a41

• [SYCL]fix ggml_sycl_mul_mat_id() to match the change of api (ggerganov#7436)

  Show description for e2b0650
  arthw

  authored
  e2b0650

  Copy full SHA for e2b0650

• github: add refactor to issue template (ggerganov#7561)

  Show description for 271ff3f
  mofosyne

  authored
  271ff3f

  Copy full SHA for 271ff3f

• llama : handle unknown utf8 bytes (ggerganov#7588)

  ggerganov

  authored
  8b99e2a

  Copy full SHA for 8b99e2a

• tests : fix test-tokenizer-0.sh

  ggerganov

  committed
  edc2943

  Copy full SHA for edc2943

• Add support for DeepseekV2ForCausalLM (ggerganov#7519)

  Show description for ee3dff6
  

  fairydreaming

  and

  sszymczy

  authored
  ee3dff6

  Copy full SHA for ee3dff6

• rpc : resource management rework (ggerganov#7562)

  Show description for 2b737ca
  rgerganov

  authored
  2b737ca

  Copy full SHA for 2b737ca

• vulkan: properly initialize vulkan devices for LLAMA_SPLIT_MODE_NONE (ggerganov#7552)

  Adriankhl

  authored
  56411a9

  Copy full SHA for 56411a9

• llama : support small Granite models (ggerganov#7481)

  Show description for 5442939
  

  giuseppe

  and

  sroecker

  authored
  5442939

  Copy full SHA for 5442939

• sycl : fix assert (ggerganov#7563)

  ggerganov

  committed
  6bd12ce

  Copy full SHA for 6bd12ce

• Tokenizer WPM fixes (ggerganov#7500)

  Show description for 02c1eca
  jaime-m-p

  authored
  02c1eca

  Copy full SHA for 02c1eca

• [SYCL] Align GEMM dispatch (ggerganov#7566)

  Show description for b864b50
  airMeng

  authored
  b864b50

  Copy full SHA for b864b50

Commits on May 29, 2024

• ggml : fix typo in ggml.c (ggerganov#7603)

  zhouwg

  authored
  504f0c3

  Copy full SHA for 504f0c3

• Add Arc A750 and Arch linux to readme-sycl.md as verified GPU model and Linux distro (ggerganov#7605)

  

  Akarshan Biswas

  authored
  0e8d8bf

  Copy full SHA for 0e8d8bf

• ggml : restore ggml_rope_xpos_inplace (ggml/0)

  Show description for 72de268
  ggerganov

  committed
  72de268

  Copy full SHA for 72de268

• sync : ggml

  ggerganov

  committed
  2ab9772

  Copy full SHA for 2ab9772

• scripts : remove mpi remnants

  ggerganov

  committed
  00281b7

  Copy full SHA for 00281b7

• ggml : use atomic_flag for critical section (ggerganov#7598)

  Show description for 87bdf2a
  slaren

  authored
  87bdf2a

  Copy full SHA for 87bdf2a

• llama-bench : add support for the RPC backend (ggerganov#7435)

  rgerganov

  authored
  210d991

  Copy full SHA for 210d991

• cuda : non-cont concat support (ggerganov#7610)

  Show description for cce3dcf
  ggerganov

  authored
  cce3dcf

  Copy full SHA for cce3dcf

• ggml : fix YARN + add tests + add asserts (ggerganov#7617)

  Show description for fb76ec3
  ggerganov

  authored
  fb76ec3

  Copy full SHA for fb76ec3

• metal : add missing asserts (ggerganov#7617)

  ggerganov

  committed
  975ec63

  Copy full SHA for 975ec63

• metal : remove invalid asserts (ggerganov#7617)

  ggerganov

  committed
  55d6226

  Copy full SHA for 55d6226

Commits on May 30, 2024

• gguf-py : Add tokenizer.ggml.pre to gguf-new-metadata.py (ggerganov#7627)

  Galunid

  authored
  eb57fee

  Copy full SHA for eb57fee

• [SYCL] fix intel docker (ggerganov#7630)

  Show description for 3854c9d
  airMeng

  authored
  3854c9d

  Copy full SHA for 3854c9d

• README: explain parallel build [no ci] (ggerganov#7618)

  JohannesGaessler

  authored
  972b555

  Copy full SHA for 972b555

• ggml : fix loongarch build (O2 issue) (ggerganov#7636)

  junchao-loongson

  authored
  d5c0582

  Copy full SHA for d5c0582

• faster avx512 exp implementation (ggerganov#7551)

  Show description for 59b0d07
  chriselrod

  authored
  59b0d07

  Copy full SHA for 59b0d07

• Move convert.py to examples/convert-legacy-llama.py (ggerganov#7430)

  Show description for 9c4c9cc
  Galunid

  authored
  9c4c9cc

  Copy full SHA for 9c4c9cc

• github: add contact links to issues and convert question into research [no ci] (ggerganov#7612)

  mofosyne

  authored
  e6157f9

  Copy full SHA for e6157f9

• readme : add Conan badge (ggerganov#7638)

  MartinDelille

  authored
  7846540

  Copy full SHA for 7846540

• Add brew installation instruction to README [no ci] (ggerganov#7616)

  makuche

  authored
  2e2340d

  Copy full SHA for 2e2340d

• Fix conan badge display [no ci] (ggerganov#7645)

  MartinDelille

  authored
  5dcdf94

  Copy full SHA for 5dcdf94

• llama : cache llama_token_to_piece (ggerganov#7587)

  Show description for 5921b8f
  ggerganov

  authored
  5921b8f

  Copy full SHA for 5921b8f

• Fixed painfully slow single process builds. (ggerganov#7326)

  Show description for 9022c33
  jboero

  authored
  9022c33

  Copy full SHA for 9022c33

• [no ci] docs: add aikit to readme (ggerganov#7650)

  Show description for 0541f06
  sozercan

  authored
  0541f06

  Copy full SHA for 0541f06

Commits on May 31, 2024

• Add convert.py removal to hot topics (ggerganov#7662)

  Galunid

  authored
  1af511f

  Copy full SHA for 1af511f

• Somehow '**' got lost (ggerganov#7663)

  Galunid

  authored
  2e32f87

  Copy full SHA for 2e32f87

• ggml : fix loongson compile warnings (ggerganov#7537)

  Show description for 0c27e6f
  

  ggerganov

  and

  junchao-loongson

  authored
  0c27e6f

  Copy full SHA for 0c27e6f

• readme : link homebrew discussion

  ggerganov

  authored
  16926df

  Copy full SHA for 16926df

• Improve HIP compatibility (ggerganov#7672)

  daniandtheweb

  authored
  30e238b

  Copy full SHA for 30e238b

• scripts: update compare_llama_bench.py [no ci] (ggerganov#7673)

  JohannesGaessler

  authored
  c8047d5

  Copy full SHA for c8047d5

• convert-hf : Handle NotImplementedError in convert-hf-to-gguf (ggerganov#7660)

  Galunid

  authored
  0515ad9

  Copy full SHA for 0515ad9

• server : update js (ggerganov#7670)

  ggerganov

  authored
  a323ec6

  Copy full SHA for a323ec6

Commits on Jun 1, 2024

• CUDA: quantized KV support for FA vec (ggerganov#7527)

  Show description for 9b59641
  JohannesGaessler

  authored
  9b59641

  Copy full SHA for 9b59641

• CUDA: fix Pascal FA, deq. KV to FP16 for batch > 8 (ggerganov#7681)

  JohannesGaessler

  authored
  750f60c

  Copy full SHA for 750f60c

• SimpleChat: Simple histogram/repeatMatching driven garbageTrimming, Settings UI, Streaming mode, OpenAi Compat (Model, Authorization Bearer), Save/Restore session, Auto Settings UI (ggerganov#7548)

  Show description for 2ac95c9
  hanishkvc

  authored
  2ac95c9

  Copy full SHA for 2ac95c9

• server : new UI (ggerganov#7633)

  Show description for 2e66683
  mounta11n

  authored
  2e66683

  Copy full SHA for 2e66683

• Fix FlashAttention debug test, FP32 assert (ggerganov#7684)

  JohannesGaessler

  authored
  e141ce6

  Copy full SHA for e141ce6

Commits on Jun 2, 2024

• [SYCL] Update rpc-server.cpp to include SYCL backend (ggerganov#7682)

  Show description for 9422c5e
  nickp27

  authored
  9422c5e

  Copy full SHA for 9422c5e

• chore : add ignore rule for generated server themes (ggerganov#7689)

  teleprint-me

  authored
  7c4e5b7

  Copy full SHA for 7c4e5b7

• flake.lock: Update (ggerganov#7686)

  Show description for 1669810
  

  ggerganov

  and

  github-actions[bot]

  authored
  1669810

  Copy full SHA for 1669810

• fix bug introduced in using calloc (ggerganov#7701)

  Show description for 3413ae2
  airlied

  authored
  3413ae2

  Copy full SHA for 3413ae2

Commits on Jun 3, 2024

• kompute : implement op_getrows_f32 (ggerganov#6403)

  Show description for 9e405b6
  woachk

  authored
  9e405b6

  Copy full SHA for 9e405b6

• llama : avoid double token-to-piece cache (ggerganov#7654)

  Show description for 549279d
  ggerganov

  authored
  549279d

  Copy full SHA for 549279d

• llama : MiniCPM support tied embeddings (ggerganov#7664)

  Show description for 6f28a33
  

  zkh2016

  and

  zhangkaihuo

  authored
  6f28a33

  Copy full SHA for 6f28a33

• cmake : add pkg-config spec file for llama.cpp (ggerganov#7702)

  andy-tai

  authored
  a10cda5

  Copy full SHA for a10cda5

• Vulkan Mixture of Experts (MoE) support (ggerganov#7628)

  Show description for 3d7ebf6
  0cc4m

  authored
  3d7ebf6

  Copy full SHA for 3d7ebf6

• make: fix debug options not being applied to NVCC (ggerganov#7714)

  JohannesGaessler

  authored
  0b832d5

  Copy full SHA for 0b832d5

• ggml : use OpenMP as a thread pool (ggerganov#7606)

  Show description for a5735e4
  

  3 peopleauthored
  a5735e4

  Copy full SHA for a5735e4

• llama : offload to RPC in addition to other backends (ggerganov#7640)

  Show description for bde7cd3
  

  rgerganov

  and

  slaren

  authored
  bde7cd3

  Copy full SHA for bde7cd3

Commits on Jun 4, 2024

• ggml : prevent builds with -ffinite-math-only (ggerganov#7726)

  Show description for 6d16169
  ggerganov

  authored
  6d16169

  Copy full SHA for 6d16169

• Per token attributes (ggerganov#7685)

  Show description for 3b38d48
  jaime-m-p

  authored
  3b38d48

  Copy full SHA for 3b38d48

• refine .gitignore (ggerganov#7688)

  Show description for b226c12
  zhouwg

  authored
  b226c12

  Copy full SHA for b226c12

• Improve hipBLAS support in CMake (ggerganov#7696)

  Show description for 987d743
  daniandtheweb

  authored
  987d743

  Copy full SHA for 987d743

• llama-bench : allow using a different printer for stderr with -oe (ggerganov#7722)

  Show description for adc9ff3
  slaren

  authored
  adc9ff3

  Copy full SHA for adc9ff3

• readme : remove obsolete Zig instructions (ggerganov#7471)

  ggerganov

  committed
  5ca0944

  Copy full SHA for 5ca0944

• llama : remove beam search (ggerganov#7736)

  ggerganov

  authored
  0cd6bd3

  Copy full SHA for 0cd6bd3

• ggml : remove OpenCL (ggerganov#7735)

  Show description for 554c247
  ggerganov

  authored
  554c247

  Copy full SHA for 554c247

• common : refactor cli arg parsing (ggerganov#7675)

  Show description for 1442677
  ggerganov

  authored
  1442677

  Copy full SHA for 1442677

• Allow number of nodes in CUDA graph to change (ggerganov#7738)

  Show description for b90dc56
  agray3

  authored
  b90dc56

  Copy full SHA for b90dc56

• Fix per token atrributes bits (ggerganov#7749)

  jaime-m-p

  authored
  c90dbe0

  Copy full SHA for c90dbe0

Commits on Jun 5, 2024

• readme : remove -ins (ggerganov#7759)

  Show description for 9973e81
  arch-btw

  authored
  9973e81

  Copy full SHA for 9973e81

• Merge branch 'layla-build' into merge

  l3utterfly

  authored
  0ac83d0

  Copy full SHA for 0ac83d0