Code

llama : Metal inference#1642

Code

Merged

ggerganov merged 49 commits intomasterggerganov/llama.cpp:masterfrom metalggerganov/llama.cpp:metalCopy head branch name to clipboardJun 4, 2023

+1,676-93

Commits

Commits on May 29, 2023

• mtl : export the LLaMA computation graph

  ggerganov

  committed
  f85020b

  Copy full SHA for f85020b

• ci : disable temporary

  ggerganov

  committed
  98c267f

  Copy full SHA for 98c267f

• mtl : adapt the MNIST example as starter

  ggerganov

  committed
  b23fe8c

  Copy full SHA for b23fe8c

• mtl : no need for mtl-export tool, add cli arg for main instead

  ggerganov

  committed
  a792cbd

  Copy full SHA for a792cbd

• mtl : export just a small part of the graph for now to make it easier

  ggerganov

  committed
  897d6d8

  Copy full SHA for 897d6d8

• mtl : move MSL code into separate file for easy editing

  ggerganov

  committed
  248a8c3

  Copy full SHA for 248a8c3

• mtl : initial get_rows_q4_0 kernel

  ggerganov

  committed
  a8fd9dc

  Copy full SHA for a8fd9dc

Commits on May 30, 2023

• mtl : confirmed get_rows_q4_0 is working correctly

  ggerganov

  committed
  794704e

  Copy full SHA for 794704e

• mtl : add rms_norm kernel + confirm working

  ggerganov

  committed
  72256eb

  Copy full SHA for 72256eb

• mtl : add mul kernel + confirm working

  ggerganov

  committed
  64afc0b

  Copy full SHA for 64afc0b

• mtl : initial mul_mat Q4 kernel (wrong results)

  ggerganov

  committed
  2a24994

  Copy full SHA for 2a24994

• mtl : mul_mat fixes (still wrong)

  ggerganov

  committed
  96d0052

  Copy full SHA for 96d0052

• mtl : another mul_mat Q4 (still does not work)

  ggerganov

  committed
  29bec00

  Copy full SHA for 29bec00

• mtl : working mul_mat q4

  ggerganov

  committed
  b2fd06c

  Copy full SHA for b2fd06c

Commits on May 31, 2023

• ggml : fix handling of "view" ops in ggml_graph_import()

  ggerganov

  committed
  6af6a05

  Copy full SHA for 6af6a05

• mtl : add rope kernel

  ggerganov

  committed
  1213af7

  Copy full SHA for 1213af7

• mtl : add reshape and transpose handling

  ggerganov

  committed
  7ca81e9

  Copy full SHA for 7ca81e9

Commits on Jun 1, 2023

• ggml : store offset as opt arg for ggml_view_xd() operators

  ggerganov

  committed
  94ea9e7

  Copy full SHA for 94ea9e7

• mtl : add cpy kernel + handle view ops

  ggerganov

  committed
  948fcfd

  Copy full SHA for 948fcfd

• mtl : confirm f16 x f32 attention mul mat

  ggerganov

  committed
  51efb59

  Copy full SHA for 51efb59

• mtl : add scale kernel

  ggerganov

  committed
  0f1c580

  Copy full SHA for 0f1c580

• mtl : add diag_mask_inf kernel

  ggerganov

  committed
  17a7036

  Copy full SHA for 17a7036

• mtl : fix soft_max kernel

  ggerganov

  committed
  17930fb

  Copy full SHA for 17930fb

• ggml : update ggml_nbytes() to handle non-contiguous tensors

  ggerganov

  committed
  f67c2d8

  Copy full SHA for f67c2d8

• mtl : verify V tensor contents

  ggerganov

  committed
  a266c26

  Copy full SHA for a266c26

• mtl : add f32 -> f32 cpy kernel

  ggerganov

  committed
  a0cc3de

  Copy full SHA for a0cc3de

• mtl : add silu kernel

  ggerganov

  committed
  42dca40

  Copy full SHA for 42dca40

• mtl : add non-broadcast mul kernel

  ggerganov

  committed
  fbd3f62

  Copy full SHA for fbd3f62

• mtl : full GPU inference of the computation graph

  ggerganov

  committed
  9665429

  Copy full SHA for 9665429

• mtl : optimize rms_norm and soft_max kernels

  ggerganov

  committed
  f0196a7

  Copy full SHA for f0196a7

• mtl : add f16 mat x f32 vec multiplication kernel

  ggerganov

  committed
  e55f7b0

  Copy full SHA for e55f7b0

Commits on Jun 2, 2023

• mtl : fix bug in f16 x f32 mul mat + speed-up computation

  ggerganov

  committed
  3367146

  Copy full SHA for 3367146

• mtl : faster mul_mat_q4_0_f32 kernel

  ggerganov

  committed
  847bbfe

  Copy full SHA for 847bbfe

• mtl : fix kernel signature + roll inner loop

  ggerganov

  committed
  70c3387

  Copy full SHA for 70c3387

• mtl : more threads for rms_norm + better timing

  ggerganov

  committed
  b088e14

  Copy full SHA for b088e14

• mtl : remove printfs from inner loop

  ggerganov

  committed
  6276057

  Copy full SHA for 6276057

• mtl : simplify implementation

  ggerganov

  committed
  03c2d72

  Copy full SHA for 03c2d72

• mtl : add save/load vocab to ggml file

  ggerganov

  committed
  640a889

  Copy full SHA for 640a889

• mtl : plug Metal inference into llama.cpp (very quick-n-dirty)

  ggerganov

  committed
  2f4e9d1

  Copy full SHA for 2f4e9d1

Commits on Jun 3, 2023

• mtl : make it work with main example

  Show description for 4df2ef3
  ggerganov

  committed
  4df2ef3

  Copy full SHA for 4df2ef3

Commits on Jun 4, 2023

• mtl : preparing for merge

  ggerganov

  committed
  18e482a

  Copy full SHA for 18e482a

• mtl : clean-up ggml mtl interface + suport scratch / inplace

  ggerganov

  committed
  e4b5222

  Copy full SHA for e4b5222

• mtl : remove temp / debug code

  ggerganov

  committed
  e26cd6b

  Copy full SHA for e26cd6b

• metal : final refactoring and simplification

  ggerganov

  committed
  a7fb899

  Copy full SHA for a7fb899

• Revert "ci : disable temporary"

  Show description for d8a7486
  ggerganov

  committed
  d8a7486

  Copy full SHA for d8a7486

• metal : add comments

  ggerganov

  committed
  b252acb

  Copy full SHA for b252acb

• metal : clean-up stuff, fix typos

  ggerganov

  committed
  db3db9e

  Copy full SHA for db3db9e

• readme : add Metal instructions

  ggerganov

  committed
  e33002d

  Copy full SHA for e33002d

• readme : add example for main

  ggerganov

  committed
  324e823

  Copy full SHA for 324e823