Code

merge from upstream#45

Code

Merged

l3utterfly merged 66 commits intolayla-buildl3utterfly/llama.cpp:layla-buildfrom mergel3utterfly/llama.cpp:mergeCopy head branch name to clipboardNov 16, 2024

+52,546-21,454

Commits

Commits on Nov 5, 2024

• llama : add <|tool_call|> formatting to Granite template (ggerganov#10177)

  Show description for b8deef0
  gabe-l-hart

  authored
  b8deef0

  Copy full SHA for b8deef0

Commits on Nov 6, 2024

• metal : add quantized FA support (ggerganov#10149)

  Show description for a1eaf6a
  ggerganov

  authored
  a1eaf6a

  Copy full SHA for a1eaf6a

• ggml : adjust is_first_call init value (ggerganov#10193)

  Show description for 1dc04b2
  ggerganov

  authored
  1dc04b2

  Copy full SHA for 1dc04b2

• metal : fix from ptr buffer name (ggerganov#10189)

  slaren

  authored
  94d8cb8

  Copy full SHA for 94d8cb8

• server : remove hack for extra parallel slot (ggerganov#10187)

  Show description for b11f9ba
  ggerganov

  authored
  b11f9ba

  Copy full SHA for b11f9ba

• metal : add BF16 support (ggerganov#8439)

  Show description for 5c333e0
  ggerganov

  authored
  5c333e0

  Copy full SHA for 5c333e0

Commits on Nov 7, 2024

• Optimize RWKV6 Operator Naming and Implement Multi-core CPU/ SYCL Acceleration (ggerganov#10133)

  Show description for 3bcd40b
  

  6 peopleauthored
  3bcd40b

  Copy full SHA for 3bcd40b

• fix q4_0_8_8 format for corrupted tokens issue (ggerganov#10198)

  Show description for 2319126
  

  snadampal

  and

  EC2 Default User

  authored
  2319126

  Copy full SHA for 2319126

• DRY: Fixes clone functionality (ggerganov#10192)

  wwoodsTM

  authored
  5107e8c

  Copy full SHA for 5107e8c

• Remove identical wte/etw logic for jais (ggerganov#10203)

  fmz

  authored
  60e17ce

  Copy full SHA for 60e17ce

• ggml : add ggml-cpu.h to the public headers (ggerganov#10204)

  slaren

  authored
  97404c4

  Copy full SHA for 97404c4

• scripts : sync update

  ggerganov

  committed
  a2c6fd7

  Copy full SHA for a2c6fd7

• sync : ggml

  ggerganov

  committed
  3b08828

  Copy full SHA for 3b08828

• scripts : add amx to sync-ggml.sh [no ci]

  ggerganov

  committed
  eec4d71

  Copy full SHA for eec4d71

• server : revamp chat UI with vuejs and daisyui (ggerganov#10175)

  Show description for a71d81c
  ngxson

  authored
  a71d81c

  Copy full SHA for a71d81c

• server : minor UI fix (ggerganov#10207)

  ngxson

  authored
  76c6e7f

  Copy full SHA for 76c6e7f

Commits on Nov 8, 2024

• swift : exclude ggml-metal-embed.metal (ggerganov#10211)

  Show description for d05b312
  jhen0409

  authored
  d05b312

  Copy full SHA for d05b312

• metal : optimize FA kernels (ggerganov#10171)

  Show description for 841f27a
  ggerganov

  authored
  841f27a

  Copy full SHA for 841f27a

• metal : improve clarity (minor) (ggerganov#10171)

  ggerganov

  committed
  695ad75

  Copy full SHA for 695ad75

• metal : opt-in compile flag for BF16 (ggerganov#10218)

  Show description for ec450d3
  ggerganov

  authored
  ec450d3

  Copy full SHA for ec450d3

Commits on Nov 9, 2024

• scripts : fix pattern and get n_tokens in one go (ggerganov#10221)

  lhpqaq

  authored
  8fc393f

  Copy full SHA for 8fc393f

• ggml : optimize llamafile cpu matrix multiplication for ppc64le (ggerganov#10156)

  Show description for e892134
  amritahs-ibm

  authored
  e892134

  Copy full SHA for e892134

• ggml: fix zero division in ‘dne’ calculation in CUDA COUNT_EQUAL operator when ‘ne’ is small (ggerganov#10213)

  SongXiaoXi

  authored
  5b359bb

  Copy full SHA for 5b359bb

• metal : hide debug messages from normal log

  ggerganov

  committed
  46323fa

  Copy full SHA for 46323fa

• llama : fix Qwen model type strings

  ggerganov

  committed
  f018acb

  Copy full SHA for f018acb

• metal : fix F32 accumulation in FA vec kernel (ggerganov#10232)

  ggerganov

  authored
  bb38cdd

  Copy full SHA for bb38cdd

• metal : fix build and some more comments (ggerganov#10229)

  ggerganov

  authored
  39a334a

  Copy full SHA for 39a334a

• metal : reorder write loop in mul mat kernel + style (ggerganov#10231)

  Show description for 6423c65
  ggerganov

  authored
  6423c65

  Copy full SHA for 6423c65

Commits on Nov 10, 2024

• vulkan: Fix newly added tests for permuted mul_mat and 1D im2col (ggerganov#10226)

  jeffbolznv

  authored
  160687b

  Copy full SHA for 160687b

• server : (web UI) Add back sampler settings (ggerganov#10239)

  Show description for 505f332
  

  MaggotHATE

  and

  ngxson

  authored
  505f332

  Copy full SHA for 505f332

• flake.lock: Update (ggerganov#10243)

  Show description for 4b3a921
  

  ggerganov

  and

  github-actions[bot]

  authored
  4b3a921

  Copy full SHA for 4b3a921

Commits on Nov 11, 2024

• server : enable KV cache defrag by default (ggerganov#10233)

  Show description for b141e5f
  ggerganov

  authored
  b141e5f

  Copy full SHA for b141e5f

• metal : more precise Q*K in FA vec kernel (ggerganov#10247)

  ggerganov

  authored
  b0cefea

  Copy full SHA for b0cefea

• vulkan: Throttle the number of shader compiles during the build step. (ggerganov#10222)

  Show description for 54ef9cf
  jeffbolznv

  authored
  54ef9cf

  Copy full SHA for 54ef9cf

Commits on Nov 13, 2024

• vulkan: Optimize contiguous copies (ggerganov#10254)

  Show description for 80dd7ff
  jeffbolznv

  authored
  80dd7ff

  Copy full SHA for 80dd7ff

• sycl : Fixes to broken builds and test-backend-ops (ggerganov#10257)

  Show description for 2e82ffa
  Alcpz

  authored
  2e82ffa

  Copy full SHA for 2e82ffa

• metadata: Detailed Dataset Authorship Metadata (ggerganov#8875)

  Show description for a0ec17b
  mofosyne

  authored
  a0ec17b

  Copy full SHA for a0ec17b

• server : fix incorrect res in validate_model_chat_template (ggerganov#10272)

  Show description for 0e712a5
  jhen0409

  authored
  0e712a5

  Copy full SHA for 0e712a5

• server : add missing docs (ggerganov#10269)

  z80maniac

  authored
  ff7fb67

  Copy full SHA for ff7fb67

• docs : update bindings list (ggerganov#10261)

  Show description for 1ee9eea
  xuegao-tzx

  authored
  1ee9eea

  Copy full SHA for 1ee9eea

• sync : ggml

  ggerganov

  committed
  5ea926d

  Copy full SHA for 5ea926d

• llama : propagate the results of graph_compute (ggerganov#9525)

  Show description for fb4a0ec
  

  Xarbirus

  and

  ggerganov

  authored
  fb4a0ec

  Copy full SHA for fb4a0ec

• vulkan: Use macros to make the mat mul pipeline creation more concise (ggerganov#10259)

  Show description for 66798e4
  jeffbolznv

  authored
  66798e4

  Copy full SHA for 66798e4

Commits on Nov 14, 2024

• vulkan: Optimize binary ops (ggerganov#10270)

  Show description for af148c9
  jeffbolznv

  authored
  af148c9

  Copy full SHA for af148c9

• speculative : fix out-of-bounds access (ggerganov#10289)

  ggerganov

  authored
  2a82891

  Copy full SHA for 2a82891

• CUDA: no -sm row for very small matrices (ggerganov#10185)

  JohannesGaessler

  authored
  4a8ccb3

  Copy full SHA for 4a8ccb3

• ggml : build backends as libraries (ggerganov#10256)

  Show description for ae8de6d
  

  3 peopleauthored
  ae8de6d

  Copy full SHA for ae8de6d

Commits on Nov 15, 2024

• backend cpu: add online flow for aarch64 Q4_0 GEMV/GEMM kernels (ggerganov#9921)

  Show description for 1607a5e
  

  chaxu01

  and

  slaren

  authored
  1607a5e

  Copy full SHA for 1607a5e

• sycl: Use syclcompat::dp4a (ggerganov#10267)

  Show description for 5a54af4
  Rbiessy

  authored
  5a54af4

  Copy full SHA for 5a54af4

• scripts : fix regex in sync [no ci]

  ggerganov

  committed
  4802ad3

  Copy full SHA for 4802ad3

• cann: dockerfile and doc adjustment (ggerganov#10302)

  Show description for 231f936
  

  noemotiovon

  and

  noemotiovon

  authored
  231f936

  Copy full SHA for 231f936

• server : (web UI) add copy button for code block, fix api key (ggerganov#10242)

  Show description for 9901068
  ngxson

  authored
  9901068

  Copy full SHA for 9901068

• sycl: Update Intel docker images to use DPC++ 2025.0 (ggerganov#10305)

  Rbiessy

  authored
  57f8355

  Copy full SHA for 57f8355

• ci: build test musa with cmake (ggerganov#10298)

  Show description for f0204a0
  yeahdongcn

  authored
  f0204a0

  Copy full SHA for f0204a0

• AVX BF16 and single scale quant optimizations (ggerganov#10212)

  Show description for 1842922
  netrunnereve

  authored
  1842922

  Copy full SHA for 1842922

• sync : ggml

  ggerganov

  committed
  cbf5541

  Copy full SHA for cbf5541

• ggml : vulkan logs (whisper/2547)

  

  thewh1teagle

  authored and

  ggerganov

  committed
  3225008

  Copy full SHA for 3225008

• cmake : fix ppc64 check (whisper/0)

  Show description for 09ecbcb
  ggerganov

  committed
  09ecbcb

  Copy full SHA for 09ecbcb

• ggml : fix some build issues

  

  slaren

  authored and

  ggerganov

  committed
  883d206

  Copy full SHA for 883d206

• scripts: update compare-llama-bench.py (ggerganov#10319)

  JohannesGaessler

  authored
  4047be7

  Copy full SHA for 4047be7

• Make updates to fix issues with clang-cl builds while using AVX512 flags (ggerganov#10314)

  Srihari-mcw

  authored
  74d73dc

  Copy full SHA for 74d73dc

Commits on Nov 16, 2024

• llama : save number of parameters and the size in llama_model (ggerganov#10286)

  Show description for 89e4caa
  FirstTimeEZ

  authored
  89e4caa

  Copy full SHA for 89e4caa

• ggml : optimize Q4_0 into Q4_0_X_Y repack (ggerganov#10324)

  eddnjjn

  authored
  1e58ee1

  Copy full SHA for 1e58ee1

• vulkan : add cmake preset debug/release (ggerganov#10306)

  FirstTimeEZ

  authored
  dd3a6ce

  Copy full SHA for dd3a6ce

• vulkan: Optimize some mat-vec mul quant shaders (ggerganov#10296)

  Show description for 772703c
  jeffbolznv

  authored
  772703c

  Copy full SHA for 772703c

• Merge branch 'layla-build' into merge

  l3utterfly

  authored
  bce287c

  Copy full SHA for bce287c