Code

Add musicgen melody#2

Code

Merged

ylacombe merged 188 commits intoadd-training-musicgenylacombe/transformers:add-training-musicgenfrom add-musicgen-melodyylacombe/transformers:add-musicgen-melodyCopy head branch name to clipboardMar 4, 2024

+31,552-5,500

Commits

Commits on Feb 7, 2024

• Add npu device for pipeline (huggingface#28885)

  Show description for 5f96855
  

  ji-huazhong

  and

  unit_test

  authored
  5f96855

  Copy full SHA for 5f96855

Commits on Feb 8, 2024

• [Docs] Fix placement of tilde character (huggingface#28913)

  Show description for 328ade8
  khipp

  authored
  328ade8

  Copy full SHA for 328ade8

• [Docs] Revert translation of '@slow' decorator (huggingface#28912)

  khipp

  authored
  33df036

  Copy full SHA for 33df036

• Fix utf-8 yaml load for marian conversion to pytorch in Windows (huggingface#28618)

  Show description for 4b236ae
  SystemPanic

  authored
  4b236ae

  Copy full SHA for 4b236ae

• [Core generation] Adds support for static KV cache (huggingface#27931)

  Show description for 115ac94
  

  4 peopleauthored
  115ac94

  Copy full SHA for 115ac94

• Remove dead TF loading code (huggingface#28926)

  Show description for 693667b
  Rocketknight1

  authored
  693667b

  Copy full SHA for 693667b

• fix: torch.int32 instead of torch.torch.int32 (huggingface#28883)

  vodkaslime

  authored
  0b693e9

  Copy full SHA for 0b693e9

• pass kwargs in stopping criteria list (huggingface#28927)

  zucchini-nlp

  authored
  cc309fd

  Copy full SHA for cc309fd

• Support batched input for decoder start ids (huggingface#28887)

  Show description for d628664
  

  zucchini-nlp

  and

  gante

  authored
  d628664

  Copy full SHA for d628664

• [Docs] Fix broken links and syntax issues (huggingface#28918)

  Show description for 2749e47
  khipp

  authored
  2749e47

  Copy full SHA for 2749e47

Commits on Feb 9, 2024

• Fix max_position_embeddings default value for llama2 to 4096 huggingface#28241 (huggingface#28754)

  Show description for de11e65
  

  karl-hajjar

  and

  amyeroberts

  authored
  de11e65

  Copy full SHA for de11e65

• Fix a wrong link to CONTRIBUTING.md section in PR template (huggingface#28941)

  B-Step62

  authored
  ebf3ea2

  Copy full SHA for ebf3ea2

• Fix type annotations on neftune_noise_alpha and fsdp_config TrainingArguments parameters (huggingface#28942)

  peblair

  authored
  d123e66

  Copy full SHA for d123e66

• [i18n-de] Translate README.md to German (huggingface#28933)

  Show description for 58e3d23
  khipp

  authored
  58e3d23

  Copy full SHA for 58e3d23

Commits on Feb 12, 2024

• [Nougat] Fix pipeline (huggingface#28242)

  Show description for f278ef2
  NielsRogge

  authored
  f278ef2

  Copy full SHA for f278ef2

• [Docs] Update README and default pipelines (huggingface#28864)

  Show description for ef5ab72
  NielsRogge

  authored
  ef5ab72

  Copy full SHA for ef5ab72

• Convert torch_dtype as str to actual torch data type (i.e. "float16" …to torch.float16) (huggingface#28208)

  Show description for cf4c20b
  

  KossaiSbai

  and

  amyeroberts

  authored
  cf4c20b

  Copy full SHA for cf4c20b

• [pipelines] updated docstring with vqa alias (huggingface#28951)

  Show description for 1709886
  cmahmut

  authored
  1709886

  Copy full SHA for 1709886

• Tests: tag test_save_load_fast_init_from_base as flaky (huggingface#28930)

  gante

  authored
  e30bbb2

  Copy full SHA for e30bbb2

• Updated requirements for image-classification samples: datasets>=2.14.0 (huggingface#28974)

  Show description for 792819f
  alekseyfa

  authored
  792819f

  Copy full SHA for 792819f

• Always initialize tied output_embeddings if it has a bias term (huggingface#28947)

  Show description for 136cd89
  hackyon

  authored
  136cd89

  Copy full SHA for 136cd89

• Clean up staging tmp checkpoint directory (huggingface#28848)

  Show description for c617f98
  woshiyyya

  authored
  c617f98

  Copy full SHA for c617f98

• [Docs] Add language identifiers to fenced code blocks (huggingface#28955)

  Show description for fe3df9d
  khipp

  authored
  fe3df9d

  Copy full SHA for fe3df9d

• [Docs] Add video section (huggingface#28958)

  Show description for 78ba9f4
  NielsRogge

  authored
  78ba9f4

  Copy full SHA for 78ba9f4

• [i18n-de] Translate CONTRIBUTING.md to German (huggingface#28954)

  Show description for d90acc1
  khipp

  authored
  d90acc1

  Copy full SHA for d90acc1

Commits on Feb 13, 2024

• [NllbTokenizer] refactor with added tokens decoder (huggingface#27717)

  Show description for b445675
  

  ArthurZucker

  and

  amyeroberts

  authored
  b445675

  Copy full SHA for b445675

• Add sudachi_projection option to BertJapaneseTokenizer (huggingface#28503)

  Show description for da20209
  

  hiroshi-matsuda-rit

  and

  ArthurZucker

  authored
  da20209

  Copy full SHA for da20209

• Static Cache: load models with MQA or GQA (huggingface#28975)

  gante

  authored
  3e70a20

  Copy full SHA for 3e70a20

• Update configuration_llama.py: fixed broken link (huggingface#28946)

  Show description for 3de6a6b
  

  AdityaKane2001

  and

  amyeroberts

  authored
  3de6a6b

  Copy full SHA for 3de6a6b

• [DETR] Update the processing to adapt masks & bboxes to reflect padding (huggingface#28363)

  Show description for bd4b83e
  amyeroberts

  authored
  bd4b83e

  Copy full SHA for bd4b83e

Commits on Feb 14, 2024

• ENH: Do not pass warning message in case quantization_config is in config but not passed as an arg (huggingface#28988)

  Show description for 1d12b8b
  

  younesbelkada

  and

  amyeroberts

  authored
  1d12b8b

  Copy full SHA for 1d12b8b

• ENH [AutoQuantizer]: enhance trainer + not supported quant methods (huggingface#28991)

  Show description for 164bdef
  younesbelkada

  authored
  164bdef

  Copy full SHA for 164bdef

• Add StableLM (huggingface#28810)

  Show description for de6029a
  jon-tow

  authored
  de6029a

  Copy full SHA for de6029a

• Add SiglipForImageClassification and CLIPForImageClassification (huggingface#28952)

  Show description for 63ffd56
  NielsRogge

  authored
  63ffd56

  Copy full SHA for 63ffd56

• AQLM quantizer support (huggingface#28928)

  Show description for 1ecf5f7
  

  5 peopleauthored
  1ecf5f7

  Copy full SHA for 1ecf5f7

• [Doc] Fix docbuilder - make BackboneMixin and BackboneConfigMixin importable from utils. (huggingface#29002)

  Show description for 7252e8d
  amyeroberts

  authored
  7252e8d

  Copy full SHA for 7252e8d

• Set the dataset format used by test_trainer to float32 (huggingface#28920)

  Show description for 69ca640
  

  ji-huazhong

  and

  unit_test

  authored
  69ca640

  Copy full SHA for 69ca640

• Introduce AcceleratorConfig dataclass (huggingface#28664)

  Show description for 0507e69
  

  muellerzr

  and

  amyeroberts

  authored
  0507e69

  Copy full SHA for 0507e69

• Fix flaky test vision encoder-decoder generate (huggingface#28923)

  zucchini-nlp

  authored
  354775b

  Copy full SHA for 354775b

• Mask Generation Task Guide (huggingface#28897)

  Show description for 3f4e79d
  

  5 peopleauthored
  3f4e79d

  Copy full SHA for 3f4e79d

• Add tie_weights() to LM heads and set bias in set_output_embeddings() (huggingface#28948)

  Show description for 725f4ad
  hackyon

  authored
  725f4ad

  Copy full SHA for 725f4ad

• Backbone kwargs in config (huggingface#28784)

  Show description for 0199a48
  

  amyeroberts

  and

  ArthurZucker

  authored
  0199a48

  Copy full SHA for 0199a48

• [TPU] Support PyTorch/XLA FSDP via SPMD (huggingface#28949)

  Show description for 5f06053
  alanwaketan

  authored
  5f06053

  Copy full SHA for 5f06053

• FIX [Trainer / tags]: Fix trainer + tags when users do not pass "tags" to trainer.push_to_hub() (huggingface#29009)

  Show description for 7a0fccc
  younesbelkada

  authored
  7a0fccc

  Copy full SHA for 7a0fccc

• [CLeanup] Revert SDPA attention changes that got in the static kv cache PR (huggingface#29027)

  Show description for 609a176
  ArthurZucker

  authored
  609a176

  Copy full SHA for 609a176

Commits on Feb 15, 2024

• Fix static generation when compiling! (huggingface#28937)

  Show description for f3788b0
  

  ArthurZucker

  and

  younesbelkada

  authored
  f3788b0

  Copy full SHA for f3788b0

• Add cuda_custom_kernel in DETA (huggingface#28989)

  Show description for 83e96dc
  SangbumChoi

  authored
  83e96dc

  Copy full SHA for 83e96dc

• DeformableDetrModel support fp16 (huggingface#29013)

  Show description for 5b6fa23
  

  DonggeunYu

  and

  amyeroberts

  authored
  5b6fa23

  Copy full SHA for 5b6fa23

• Fix copies between DETR and DETA (huggingface#29037)

  amyeroberts

  authored
  8a0ed0a

  Copy full SHA for 8a0ed0a

• FIX: Fix error with logger.warning + inline with recent refactor (huggingface#29039)

  Show description for 6d1f545
  younesbelkada

  authored
  6d1f545

  Copy full SHA for 6d1f545

• Patch to skip failing test_save_load_low_cpu_mem_usage tests (huggingface#29043)

  Show description for 4156f51
  amyeroberts

  authored
  4156f51

  Copy full SHA for 4156f51

• Removed obsolete attribute setting for AQLM quantization. (huggingface#29034)

  Show description for b0a7f44
  

  Andrei Panferov

  authored
  b0a7f44

  Copy full SHA for b0a7f44

• Fix a tiny typo in generation/utils.py::GenerateEncoderDecoderOutput's docstring (huggingface#29044)

  Show description for f3aa7db
  sadra-barikbin

  authored
  f3aa7db

  Copy full SHA for f3aa7db

Commits on Feb 16, 2024

• add test marker to run all tests with @require_bitsandbytes (huggingface#28278)

  Titus-von-Koeller

  authored
  1e402b9

  Copy full SHA for 1e402b9

• Update all references to canonical models (huggingface#29001)

  Show description for f497f56
  LysandreJik

  authored
  f497f56

  Copy full SHA for f497f56

• Update important model list (huggingface#29019)

  LysandreJik

  authored
  8876ce8

  Copy full SHA for 8876ce8

• Fix max_length criteria when using inputs_embeds (huggingface#28994)

  Show description for aee11fe
  

  3 peopleauthored
  aee11fe

  Copy full SHA for aee11fe

• Support : Leverage Accelerate for object detection/segmentation models (huggingface#28312)

  Show description for 0eb4085
  Tanmaypatil123

  authored
  0eb4085

  Copy full SHA for 0eb4085

• fix num_assistant_tokens with heuristic schedule (huggingface#28759)

  Show description for 258da40
  

  4 peopleauthored
  258da40

  Copy full SHA for 258da40

• fix failing trainer ds tests (huggingface#29057)

  pacman100

  authored
  b262808

  Copy full SHA for b262808

• auto_find_batch_size isn't yet supported with DeepSpeed/FSDP. Raise error accrodingly. (huggingface#29058)

  Show description for 4c18ddb
  pacman100

  authored
  4c18ddb

  Copy full SHA for 4c18ddb

• Honor trust_remote_code for custom tokenizers (huggingface#28854)

  Show description for be42c24
  

  rl337

  and

  Rocketknight1

  authored
  be42c24

  Copy full SHA for be42c24

• Feature: Option to set the tracking URI for MLflowCallback. (huggingface#29032)

  Show description for 161fe42
  seanswyi

  authored
  161fe42

  Copy full SHA for 161fe42

• Fix trainer test wrt DeepSpeed + auto_find_bs (huggingface#29061)

  Show description for 636b032
  

  muellerzr

  and

  amyeroberts

  authored
  636b032

  Copy full SHA for 636b032

• Add chat support to text generation pipeline (huggingface#28945)

  Show description for 2f1003b
  

  Rocketknight1

  and

  amyeroberts

  authored
  2f1003b

  Copy full SHA for 2f1003b

• [Docs] Spanish translation of task_summary.md (huggingface#28844)

  Show description for ce4fff0
  aaronjimv

  authored
  ce4fff0

  Copy full SHA for ce4fff0

Commits on Feb 19, 2024

• [Awq] Add peft support for AWQ (huggingface#28987)

  Show description for 864c8e6
  

  younesbelkada

  and

  amyeroberts

  authored
  864c8e6

  Copy full SHA for 864c8e6

• FIX [bnb / tests]: Fix currently failing bnb tests (huggingface#29092)

  Show description for a75a6c9
  younesbelkada

  authored
  a75a6c9

  Copy full SHA for a75a6c9

• fix the post-processing link (huggingface#29091)

  Show description for 593230f
  davies-w

  authored
  593230f

  Copy full SHA for 593230f

• Fix the bert-base-cased tokenizer configuration test (huggingface#29105)

  Show description for 9830858
  LysandreJik

  authored
  9830858

  Copy full SHA for 9830858

• Fix a typo in examples/pytorch/text-classification/run_classification.py (huggingface#29072)

  Ja1Zhou

  authored
  79132d4

  Copy full SHA for 79132d4

• change version (huggingface#29097)

  Show description for b2724d7
  

  ArthurZucker

  and

  ydshieh

  authored
  b2724d7

  Copy full SHA for b2724d7

• [Docs] Add resources (huggingface#28705)

  Show description for 07e3454
  

  NielsRogge

  and

  amyeroberts

  authored
  07e3454

  Copy full SHA for 07e3454

• ENH: added new output_logits option to generate function (huggingface#28667)

  Show description for 08cd694
  

  mbaak

  and

  kx79wq

  authored
  08cd694

  Copy full SHA for 08cd694

• Bnb test fix for different hardwares (huggingface#29066)

  Show description for 5ce90f3
  

  Titus-von-Koeller

  and

  younesbelkada

  authored
  5ce90f3

  Copy full SHA for 5ce90f3

• Fix two tiny typos in pipelines/base.py::Pipeline::_sanitize_parameters()'s docstring (huggingface#29102)

  Show description for a4851d9
  sadra-barikbin

  authored
  a4851d9

  Copy full SHA for a4851d9

• storing & logging gradient norm in trainer (huggingface#27326)

  Show description for 4f09d0f
  shijie-wu

  authored
  4f09d0f

  Copy full SHA for 4f09d0f

Commits on Feb 20, 2024

• Fixed nll with label_smoothing to just nll (huggingface#28708)

  Show description for 49c0b29
  nileshkokane01

  authored
  49c0b29

  Copy full SHA for 49c0b29

• [gradient_checkpointing] default to use it for torch 2.3 (huggingface#28538)

  Show description for 9094abe
  ArthurZucker

  authored
  9094abe

  Copy full SHA for 9094abe

• Move misplaced line (huggingface#29117)

  Show description for a7ff2f2
  kno10

  authored
  a7ff2f2

  Copy full SHA for a7ff2f2

• FEAT [Trainer / bnb]: Add RMSProp from bitsandbytes to HF Trainer (huggingface#29082)

  Show description for f7ef7ce
  

  younesbelkada

  and

  amyeroberts

  authored
  f7ef7ce

  Copy full SHA for f7ef7ce

• Abstract image processor arg checks. (huggingface#28843)

  Show description for 1c9134f
  

  molbap

  and

  amyeroberts

  authored
  1c9134f

  Copy full SHA for 1c9134f

• FIX [bnb / tests] Propagate the changes from huggingface#29092 to 4-bit tests (huggingface#29122)

  Show description for ff76e7c
  younesbelkada

  authored
  ff76e7c

  Copy full SHA for ff76e7c

• Llama: fix batched generation (huggingface#29109)

  gante

  authored
  7d312ad

  Copy full SHA for 7d312ad

• Generate: unset GenerationConfig parameters do not raise warning (huggingface#29119)

  gante

  authored
  a7755d2

  Copy full SHA for a7755d2

• [cuda kernels] only compile them when initializing (huggingface#29133)

  Show description for 5e95dca
  ArthurZucker

  authored
  5e95dca

  Copy full SHA for 5e95dca

• FIX [PEFT / Trainer ] Handle better peft + quantized compiled models (huggingface#29055)

  Show description for efdd436
  younesbelkada

  authored
  efdd436

  Copy full SHA for efdd436

• [Core tokenization] add_dummy_prefix_space option to help with latest issues (huggingface#28010)

  Show description for 15cfe38
  ArthurZucker

  authored
  15cfe38

  Copy full SHA for 15cfe38

• Revert low cpu mem tie weights (huggingface#29135)

  Show description for 0996a10
  amyeroberts

  authored
  0996a10

  Copy full SHA for 0996a10

• Add support for fine-tuning CLIP-like models using contrastive-image-text example (huggingface#29070)

  Show description for ee3af60
  tjs-intel

  authored
  ee3af60

  Copy full SHA for ee3af60

• Save (circleci) cache at the end of a job (huggingface#29141)

  Show description for 7688d8d
  ydshieh

  authored
  7688d8d

  Copy full SHA for 7688d8d

• [Phi] Add support for sdpa (huggingface#29108)

  hackyon

  authored
  b8b1647

  Copy full SHA for b8b1647

• Raise unused kwargs image processor (huggingface#29063)

  Show description for 1c81132
  molbap

  authored
  1c81132

  Copy full SHA for 1c81132

• Generate: missing generation config eos token setting in encoder-decoder tests (huggingface#29146)

  gante

  authored
  857fd8e

  Copy full SHA for 857fd8e

• Added image_captioning version in es and included in toctree file (huggingface#29104)

  Show description for 3c00b88
  gisturiz

  authored
  3c00b88

  Copy full SHA for 3c00b88

• Fix drop path being ignored in DINOv2 (huggingface#29147)

  Show description for c47576c
  fepegar

  authored
  c47576c

  Copy full SHA for c47576c

• [pipeline] Add pool option to image feature extraction pipeline (huggingface#28985)

  Show description for e770f03
  amyeroberts

  authored
  e770f03

  Copy full SHA for e770f03

Commits on Feb 21, 2024

• v4.39.dev.0

  ArthurZucker

  committed
  1a77f07

  Copy full SHA for 1a77f07

• 🚨 Llama: update rope scaling to match static cache changes (huggingface#29143)

  gante

  authored
  3994fa5

  Copy full SHA for 3994fa5

• torch.compile compatibility with generate + static cache (huggingface#29114)

  Show description for cc4a664
  

  fxmarty

  and

  ArthurZucker

  authored
  cc4a664

  Copy full SHA for cc4a664

• support SDPA Attention in stablelm (huggingface#29106)

  Show description for 1d0ea7a
  

  3 peopleauthored
  1d0ea7a

  Copy full SHA for 1d0ea7a

• [Maskformer] safely get backbone config (huggingface#29166)

  Show description for 58245ba
  amyeroberts

  authored
  58245ba

  Copy full SHA for 58245ba

• [ gemma] Adds support for Gemma 💎 (huggingface#29167)

  Show description for 594c127
  

  6 peopleauthored
  594c127

  Copy full SHA for 594c127

• FIX [Gemma] Fix bad rebase with transformers main (huggingface#29170)

  Show description for ae49b21
  younesbelkada

  authored
  ae49b21

  Copy full SHA for ae49b21

• Add training version check for AQLM quantizer. (huggingface#29142)

  Show description for fc37f38
  BlackSamorez

  authored
  fc37f38

  Copy full SHA for fc37f38

Commits on Feb 22, 2024

• [Gemma] Fix eager attention (huggingface#29187)

  Show description for 2a9b1f8
  sanchit-gandhi

  authored
  2a9b1f8

  Copy full SHA for 2a9b1f8

• [Mistral, Mixtral] Improve docs (huggingface#29084)

  Show description for dabe855
  NielsRogge

  authored
  dabe855

  Copy full SHA for dabe855

• Fix torch.compile with fullgraph=True when attention_mask input is used (huggingface#29211)

  Show description for 2cc8cf6
  fxmarty

  authored
  2cc8cf6

  Copy full SHA for 2cc8cf6

Commits on Feb 23, 2024

• fix(mlflow): check mlflow version to use the synchronous flag (huggingface#29195)

  Show description for 4524494
  cchen-dialpad

  authored
  4524494

  Copy full SHA for 4524494

• Fix missing translation in README_ru (huggingface#29054)

  Show description for 75ed76e
  

  strikoder

  and

  MKhalusova

  authored
  75ed76e

  Copy full SHA for 75ed76e

• Improve _update_causal_mask performance (huggingface#29210)

  Show description for 3f60d11
  alessandropalla

  authored
  3f60d11

  Copy full SHA for 3f60d11

• [Doc] update model doc qwen2 (huggingface#29238)

  Show description for 89c6481
  

  ArthurZucker

  and

  younesbelkada

  authored
  89c6481

  Copy full SHA for 89c6481

• Allow remote code repo names to contain "." (huggingface#29175)

  Show description for 371b572
  Rocketknight1

  authored
  371b572

  Copy full SHA for 371b572

• Use torch 2.2 for daily CI (model tests) (huggingface#29208)

  Show description for c8d9840
  ydshieh

  authored
  c8d9840

  Copy full SHA for c8d9840

Commits on Feb 26, 2024

• Cache is_vision_available result (huggingface#29280)

  Show description for 9fe3608
  

  Benjamin Muskalla

  authored
  9fe3608

  Copy full SHA for 9fe3608

• Use DS_DISABLE_NINJA=1 (huggingface#29290)

  Show description for 93f8617
  ydshieh

  authored
  93f8617

  Copy full SHA for 93f8617

• Add non_device_test pytest mark to filter out non-device tests (huggingface#29213)

  Show description for 2a7746c
  fxmarty

  authored
  2a7746c

  Copy full SHA for 2a7746c

• Add feature extraction mapping for automatic metadata update (huggingface#28944)

  Show description for 7c4995f
  

  merveenoyan

  and

  amyeroberts

  authored
  7c4995f

  Copy full SHA for 7c4995f

• Merge branch 'main' into add-musicgen-melody

  ylacombe

  authored
  755960a

  Copy full SHA for 755960a

• Use torch.bool instead of torch.int64 for non-persistant causal mask buffer (huggingface#29241)

  Show description for 24d59c7
  fxmarty

  authored
  24d59c7

  Copy full SHA for 24d59c7

• Generate: v4.38 removals and related updates (huggingface#29171)

  gante

  authored
  ece1b62

  Copy full SHA for ece1b62

• Track each row separately for stopping criteria (huggingface#29116)

  zucchini-nlp

  authored
  8f2f0f0

  Copy full SHA for 8f2f0f0

• [docs] Spanish translation of tasks_explained.md (huggingface#29224)

  Show description for 9f7535b
  aaronjimv

  authored
  9f7535b

  Copy full SHA for 9f7535b

• [i18n-zh] Translated torchscript.md into Chinese (huggingface#29234)

  Show description for b433404
  windsonsea

  authored
  b433404

  Copy full SHA for b433404

• 🌐 [i18n-ZH] Translate chat_templating.md into Chinese (huggingface#28790)

  Show description for 734eb25
  

  6 peopleauthored
  734eb25

  Copy full SHA for 734eb25

• [i18n-vi] Translate README.md to Vietnamese (huggingface#29229)

  Show description for c291350
  hoangsvit

  authored
  c291350

  Copy full SHA for c291350

• [i18n-zh] Translated task/asr.md into Chinese (huggingface#29233)

  Show description for a44d2dc
  windsonsea

  authored
  a44d2dc

  Copy full SHA for a44d2dc

• Fixed Deformable Detr typo when loading cuda kernels for MSDA (huggingface#29294)

  EduardoPach

  authored
  3b8c053

  Copy full SHA for 3b8c053

• Adding SegGPT (huggingface#27735)

  Show description for 3fcfbe7
  

  4 peopleauthored
  3fcfbe7

  Copy full SHA for 3fcfbe7

Commits on Feb 27, 2024

• GenerationConfig validate both constraints and force_words_ids (huggingface#29163)

  Show description for 871ba71
  FredericOdermatt

  authored
  871ba71

  Copy full SHA for 871ba71

• Add generate kwargs to VQA pipeline (huggingface#29134)

  regisss

  authored
  a3f9221

  Copy full SHA for a3f9221

• Cleaner Cache dtype and device extraction for CUDA graph generation for quantizers compatibility (huggingface#29079)

  Show description for e3fc90a
  BlackSamorez

  authored
  e3fc90a

  Copy full SHA for e3fc90a

• Image Feature Extraction docs (huggingface#28973)

  Show description for 83e366b
  

  3 peopleauthored
  83e366b

  Copy full SHA for 83e366b

• Fix attn_implementation documentation (huggingface#29295)

  Show description for 6d3b643
  fxmarty

  authored
  6d3b643

  Copy full SHA for 6d3b643

• [tests] enable benchmark unit tests on XPU (huggingface#29284)

  Show description for 63a0c8f
  faaany

  authored
  63a0c8f

  Copy full SHA for 63a0c8f

• Use torch 2.2 for deepspeed CI (huggingface#29246)

  Show description for 5c341d4
  ydshieh

  authored
  5c341d4

  Copy full SHA for 5c341d4

• Add compatibility with skip_memory_metrics for mps device (huggingface#29264)

  Show description for 8a1faf2
  SunMarc

  authored
  8a1faf2

  Copy full SHA for 8a1faf2

• Token level timestamps for long-form generation in Whisper (huggingface#29148)

  zucchini-nlp

  authored
  ddf7ac4

  Copy full SHA for ddf7ac4

• Fix a few typos in GenerationMixin's docstring (huggingface#29277)

  Show description for 227cd54
  

  sadra-barikbin

  and

  gante

  authored
  227cd54

  Copy full SHA for 227cd54

• [i18n-zh] Translate fsdp.md into Chinese (huggingface#29305)

  Show description for 83ab011
  windsonsea

  authored
  83ab011

  Copy full SHA for 83ab011

Commits on Feb 28, 2024

• Starcoder2 model - bis (huggingface#29215)

  Show description for 63caa37
  

  4 peopleauthored
  63caa37

  Copy full SHA for 63caa37

• simplify get_class_in_module and fix for paths containing a dot (huggingface#29262)

  cebtenzzre

  authored
  bd5b986

  Copy full SHA for bd5b986

• FIX [Gemma / CI] Make sure our runners have access to the model (huggingface#29242)

  Show description for ad00c48
  

  younesbelkada

  and

  ArthurZucker

  authored
  ad00c48

  Copy full SHA for ad00c48

• Remove numpy usage from owlvit (huggingface#29326)

  Show description for e715c78
  fxmarty

  authored
  e715c78

  Copy full SHA for e715c78

• [require_read_token] fix typo (huggingface#29345)

  Show description for a528885
  ArthurZucker

  authored
  a528885

  Copy full SHA for a528885

• [T5 and Llama Tokenizer] remove warning (huggingface#29346)

  Show description for 7c87f35
  

  ArthurZucker

  and

  hiaoxui

  authored
  7c87f35

  Copy full SHA for 7c87f35

• [Llama ROPE] Fix torch export but also slow downs in forward (huggingface#29198)

  Show description for 8a8a0a4
  ArthurZucker

  authored
  8a8a0a4

  Copy full SHA for 8a8a0a4

• Disable Mixtral output_router_logits during inference (huggingface#29249)

  Show description for 2ce56d3
  LeonardoEmili

  authored
  2ce56d3

  Copy full SHA for 2ce56d3

• Idefics: generate fix (huggingface#29320)

  gante

  authored
  7628b3a

  Copy full SHA for 7628b3a

• RoPE loses precision for Llama / Gemma + Gemma logits.float() (huggingface#29285)

  Show description for d3a4b47
  

  danielhanchen

  and

  ArthurZucker

  authored
  d3a4b47

  Copy full SHA for d3a4b47

• check if position_ids exists before using it (huggingface#29306)

  Show description for 554e7ad
  

  jiqing-feng

  and

  gante

  authored
  554e7ad

  Copy full SHA for 554e7ad

• [CI] Quantization workflow (huggingface#29046)

  Show description for f54d82c
  

  SunMarc

  and

  younesbelkada

  authored
  f54d82c

  Copy full SHA for f54d82c

• Better SDPA unmasking implementation (huggingface#29318)

  Show description for 49204c1
  fxmarty

  authored
  49204c1

  Copy full SHA for 49204c1

• [i18n-zh] Sync source/zh/index.md (huggingface#29331)

  Show description for 2209b7a
  windsonsea

  authored
  2209b7a

  Copy full SHA for 2209b7a

Commits on Feb 29, 2024

• FIX [CI / starcoder2] Change starcoder2 path to correct one for slow tests (huggingface#29359)

  Show description for 1aee9af
  younesbelkada

  authored
  1aee9af

  Copy full SHA for 1aee9af

• FIX [CI]: Fix failing tests for peft integration (huggingface#29330)

  Show description for 8d8ac9c
  younesbelkada

  authored
  8d8ac9c

  Copy full SHA for 8d8ac9c

• FIX [CI] require_read_token in the llama FA2 test (huggingface#29361)

  Show description for b647acd
  younesbelkada

  authored
  b647acd

  Copy full SHA for b647acd

• Avoid using uncessary get_values(MODEL_MAPPING) (huggingface#29362)

  Show description for 44fe1a1
  ydshieh

  authored
  44fe1a1

  Copy full SHA for 44fe1a1

• Patch YOLOS and others (huggingface#29353)

  Show description for bb4f816
  NielsRogge

  authored
  bb4f816

  Copy full SHA for bb4f816

• Fix @require_read_token in tests (huggingface#29367)

  Wauplin

  authored
  0ad770c

  Copy full SHA for 0ad770c

Commits on Mar 1, 2024

• Expose offload_buffers parameter of accelerate to PreTrainedModel.from_pretrained method (huggingface#28755)

  Show description for 5ee0868
  notsyncing

  authored
  5ee0868

  Copy full SHA for 5ee0868

• Fix Base Model Name of LlamaForQuestionAnswering (huggingface#29258)

  Show description for 2858d6c
  lenglaender

  authored
  2858d6c

  Copy full SHA for 2858d6c

• FIX [quantization / ESM] Fix ESM 8bit / 4bit with bitsandbytes (huggingface#29329)

  Show description for 50db7ca
  

  younesbelkada

  and

  ArthurZucker

  authored
  50db7ca

  Copy full SHA for 50db7ca

• [Llama + AWQ] fix prepare_inputs_for_generation 🫠 (huggingface#29381)

  Show description for e7b9837
  ArthurZucker

  authored
  e7b9837

  Copy full SHA for e7b9837

• 🚨🚨[Whisper Tok] Update integration test (huggingface#29368)

  Show description for 0a0a279
  sanchit-gandhi

  authored
  0a0a279

  Copy full SHA for 0a0a279

• [YOLOS] Fix - return padded annotations (huggingface#29300)

  Show description for f1b1379
  amyeroberts

  authored
  f1b1379

  Copy full SHA for f1b1379

• Support subfolder with AutoProcessor (huggingface#29169)

  Show description for 15f8296
  JingyaHuang

  authored
  15f8296

  Copy full SHA for 15f8296

• Fix llama + gemma accelete tests (huggingface#29380)

  SunMarc

  authored
  cec7733

  Copy full SHA for cec7733

• Fix deprecated arg issue (huggingface#29372)

  Show description for 1a7c117
  muellerzr

  authored
  1a7c117

  Copy full SHA for 1a7c117

• Correct zero division error in inverse sqrt scheduler (huggingface#28982)

  Show description for 831bc25
  DavidAfonsoValente

  authored
  831bc25

  Copy full SHA for 831bc25

Commits on Mar 4, 2024

• [tests] enable automatic speech recognition pipeline tests on XPU (huggingface#29308)

  Show description for aade711
  faaany

  authored
  aade711

  Copy full SHA for aade711

• update path to hub files in the error message (huggingface#29369)

  Show description for 38953a7
  poedator

  authored
  38953a7

  Copy full SHA for 38953a7

• [Mixtral] Fixes attention masking in the loss (huggingface#29363)

  Show description for 39ef3fb
  

  DesmonDay

  and

  KB-Ding

  authored
  39ef3fb

  Copy full SHA for 39ef3fb

• Add mlx support to BatchEncoding.convert_to_tensors (huggingface#29406)

  Show description for 704b3f7
  tidely

  authored
  704b3f7

  Copy full SHA for 704b3f7

• Workaround for huggingface#27758 to avoid ZeroDivisionError (huggingface#28756)

  tleyden

  authored
  c38a122

  Copy full SHA for c38a122

• Convert SlimSAM checkpoints (huggingface#28379)

  Show description for 5e4b69d
  

  NielsRogge

  and

  ArthurZucker

  authored
  5e4b69d

  Copy full SHA for 5e4b69d

• Fix: Fixed the previous tracking URI setting logic to prevent clashes with original MLflow code. (huggingface#29096)

  Show description for 81220cb
  

  seanswyi

  and

  amyeroberts

  authored
  81220cb

  Copy full SHA for 81220cb

• Fix OneFormer post_process_instance_segmentation for panoptic tasks (huggingface#29304)

  Show description for 8ef9862
  nickthegroot

  authored
  8ef9862

  Copy full SHA for 8ef9862

• 🚨 Fully revert atomic checkpointing 🚨 (huggingface#29370)

  Show description for 1681a6d
  muellerzr

  authored
  1681a6d

  Copy full SHA for 1681a6d

• Fix grad_norm unserializable tensor log failure (huggingface#29212)

  Show description for 7941769
  svenschultze

  authored
  7941769

  Copy full SHA for 7941769

• Avoid edge case in audio utils (huggingface#28836)

  ylacombe

  authored
  bcd23a5

  Copy full SHA for bcd23a5

• Merge branch 'main' into add-musicgen-melody

  ylacombe

  authored
  8b9177f

  Copy full SHA for 8b9177f

• replace musicgen melody checkpoints with official orga

  ylacombe

  committed
  ad26dc9

  Copy full SHA for ad26dc9

• rename ylacombe->facebook in checkpoints

  ylacombe

  committed
  7595256

  Copy full SHA for 7595256

• fix copies

  ylacombe

  committed
  2576806

  Copy full SHA for 2576806

• remove unecessary warning

  ylacombe

  committed
  379d70b

  Copy full SHA for 379d70b

• add shape in code docstrings

  ylacombe

  committed
  9795c6f

  Copy full SHA for 9795c6f

• add files to slow doc tests

  ylacombe

  committed
  b03b36d

  Copy full SHA for b03b36d