Code

Add sdpa and FA2 for CLIP#31940

Code

Merged

sayakpaul merged 19 commits intohuggingface:mainhuggingface/transformers:mainfrom qubvel:add-clip-sdpa-fa2qubvel/transformers:add-clip-sdpa-fa2Copy head branch name to clipboardJul 18, 2024

+682-44

Commits

Commits on Jul 17, 2024

• Squashed commit of the following:

  Show description for 0d1fa8e
  qubvel

  committed
  0d1fa8e

  Copy full SHA for 0d1fa8e

• Remove fallback for empty attention mask (expensive operation)

  qubvel

  committed
  cca2ec6

  Copy full SHA for cca2ec6

• Fix typing in copies

  qubvel

  committed
  f3d8024

  Copy full SHA for f3d8024

• Add flash attention

  qubvel

  committed
  0bbc912

  Copy full SHA for 0bbc912

• Add flash attention tests

  qubvel

  committed
  06c1cea

  Copy full SHA for 06c1cea

• List CLIP in FA docs

  qubvel

  committed
  650aaa5

  Copy full SHA for 650aaa5

• Fix embeddings attributes and tf

  qubvel

  committed
  fa01bb6

  Copy full SHA for fa01bb6

• [run-slow] clip

  qubvel

  committed
  60207fe

  Copy full SHA for 60207fe

• Update clip documentation

  qubvel

  committed
  df54b52

  Copy full SHA for df54b52

• Remove commented code, skip compile dynamic for CLIPModel

  qubvel

  committed
  e5c8aae

  Copy full SHA for e5c8aae

• Fix doc

  qubvel

  committed
  8376f43

  Copy full SHA for 8376f43

• Fix doc 2

  qubvel

  committed
  088a4ec

  Copy full SHA for 088a4ec

• Remove double transpose

  qubvel

  committed
  9028968

  Copy full SHA for 9028968

• Add torch version check for contiguous()

  qubvel

  committed
  ec859fc

  Copy full SHA for ec859fc

• Add comment to test mixin

  qubvel

  committed
  778ef65

  Copy full SHA for 778ef65

• Fix copies

  qubvel

  committed
  1528708

  Copy full SHA for 1528708

• Add comment for mask

  qubvel

  committed
  de681a5

  Copy full SHA for de681a5

• Update docs

  qubvel

  committed
  cfc5240

  Copy full SHA for cfc5240

• [run-slow] clip

  qubvel

  committed
  d2328d3

  Copy full SHA for d2328d3