Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Loading GGUF support #2

Draft
wants to merge 253 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
253 commits
Select commit Hold shift + click to select a range
21c912e
Fix config + attn_implementation in AutoModelForCausalLM.from_pretrai…
hiyouga Apr 19, 2024
d2cec09
Add TF swiftformer (#23342)
joaocmd Apr 19, 2024
8c12690
[Grounding DINO] Add resources (#30232)
NielsRogge Apr 19, 2024
81e4324
add q2_k q3_k q5_k support from @99991
younesbelkada Apr 22, 2024
8a0d5b8
fix tests
younesbelkada Apr 22, 2024
b20b017
Nits for model docs (#29795)
merveenoyan Apr 22, 2024
8b02bb6
Enable multi-device for more models (#30379)
jla524 Apr 22, 2024
9138935
GenerationConfig: warn if pad token is negative (#30187)
zucchini-nlp Apr 22, 2024
f16caf4
Add FSDP config for CPU RAM efficient loading through accelerate (#30…
helloworld1 Apr 22, 2024
08534f3
Update doc
LysandreJik Apr 22, 2024
ebd9944
Style
LysandreJik Apr 22, 2024
2d92db8
`Llama` family, fix `use_cache=False` generation (#30380)
ArthurZucker Apr 22, 2024
0e9d44d
Update docstrings for text generation pipeline (#30343)
Rocketknight1 Apr 22, 2024
0d84901
Terminator strings for generate() (#28932)
Rocketknight1 Apr 22, 2024
5c913ec
Docs
LysandreJik Apr 22, 2024
f3b3533
Fix layerwise GaLore optimizer hard to converge with warmup scheduler…
hiyouga Apr 22, 2024
6c7335e
Jamba: fix left-padding test (#30389)
gante Apr 22, 2024
13b3b90
Fix DETA save_pretrained (#30326)
qubvel Apr 22, 2024
367a0db
FIX / PEFT: Pass device correctly to peft (#30397)
younesbelkada Apr 22, 2024
8b81bfb
Merge remote-tracking branch 'upstream/main' into HEAD
younesbelkada Apr 22, 2024
c49f1a8
fix CI
younesbelkada Apr 22, 2024
569743f
Add sdpa and fa2 the Wav2vec2 family. (#30121)
kamilakesbi Apr 22, 2024
7fa538b
Update docs/source/en/gguf.md
younesbelkada Apr 22, 2024
5485327
Update docs/source/en/gguf.md
younesbelkada Apr 22, 2024
b4c18a8
[FEAT]: EETQ quantizer support (#30262)
dtlzhuangz Apr 22, 2024
e74d793
[docs] LLM inference (#29791)
stevhliu Apr 22, 2024
6c1295a
show `-rs` to show skip reasons (#30318)
ArthurZucker Apr 23, 2024
408453b
Add inputs embeds in generation (#30269)
zucchini-nlp Apr 23, 2024
c651ea9
[Grounding DINO] Add support for cross-attention in GroundingDinoMult…
EduardoPach Apr 23, 2024
179ab09
remove redundant logging from longformer (#30365)
riklopfer Apr 23, 2024
04ac324
fix: link to HF repo/tree/revision when a file is missing (#30406)
mapmeld Apr 23, 2024
2d61823
[tests] add `require_torch_sdpa` for test that needs sdpa support (#3…
faaany Apr 23, 2024
31921d8
Jax: scipy version pin (#30402)
gante Apr 23, 2024
074f05e
Merge branch 'main' into gguf-support
younesbelkada Apr 23, 2024
ca8363e
Compute merges
LysandreJik Apr 23, 2024
77b59dc
Fix on "cache position" for assisted generation (#30068)
zucchini-nlp Apr 23, 2024
57fc00f
fix for itemsize => element_size() for torch backwards compat (#30133)
winglian Apr 23, 2024
4b63d01
Make EosTokenCriteria compatible with mps (#30376)
pcuenca Apr 23, 2024
d179b9d
FIX: re-add bnb on docker image (#30427)
younesbelkada Apr 23, 2024
416fdba
Fix LayoutLMv2 init issue and doctest (#30278)
ydshieh Apr 23, 2024
696eded
Remove old TF port docs (#30426)
Rocketknight1 Apr 23, 2024
b8b1e44
Rename torch.run to torchrun (#30405)
steven-safeai Apr 23, 2024
12c39e5
Fix use_cache for xla fsdp (#30353)
alanwaketan Apr 23, 2024
e34da3e
[`LlamaTokenizerFast`] Refactor default llama (#28881)
ArthurZucker Apr 23, 2024
0eb8fbc
Remove task guides auto-update in favor of links towards task pages (…
LysandreJik Apr 24, 2024
d4e92f1
Remove add-new-model in favor of add-new-model-like (#30424)
LysandreJik Apr 24, 2024
c6bba94
Remove mentions of models in the READMEs and link to the documentatio…
LysandreJik Apr 24, 2024
fc34f84
New model PR needs green (slow tests) CI (#30341)
ydshieh Apr 24, 2024
89c510d
Add llama3 (#30334)
ArthurZucker Apr 24, 2024
767e351
Fix YOLOS image processor resizing (#30436)
qubvel Apr 24, 2024
9a4a119
[`Llava`] + CIs fix red cis and llava integration tests (#30440)
ArthurZucker Apr 24, 2024
16c8e17
[tests] make test device-agnostic (#30444)
faaany Apr 24, 2024
5d64ae9
fix uncaught init of linear layer in clip's/siglip's for image classi…
vasqu Apr 24, 2024
37fa1f6
fix jamba slow foward for multi-gpu (#30418)
SunMarc Apr 24, 2024
d26c141
[SegGPT] Fix loss calculation (#30421)
EduardoPach Apr 24, 2024
42fed15
Add `paths` filter to avoid the chance of being triggered (#30453)
ydshieh Apr 24, 2024
c9693db
Phi-3 (#30423)
gugarosa Apr 24, 2024
d0d430f
Fix wrong indent in `utils/check_if_new_model_added.py` (#30456)
ydshieh Apr 24, 2024
661190b
[`research_project`] Most of the security issues come from this requi…
ArthurZucker Apr 24, 2024
d1d94d7
Neuron: When save_safetensor=False, no need to move model to CPU (#29…
jeffhataws Apr 24, 2024
5c57463
Enable fp16 on CPU (#30459)
muellerzr Apr 24, 2024
6ad9c8f
Non blocking support to torch DL's (#30465)
muellerzr Apr 24, 2024
fbb41cd
consistent job / pytest report / artifact name correspondence (#30392)
ydshieh Apr 24, 2024
2a0c9b0
Merge branch 'main' into gguf-support
younesbelkada Apr 25, 2024
cebb072
Workflow / ENH: Add SSH into our runners workflow (#30425)
younesbelkada Apr 25, 2024
cd0cd12
FIX / Workflow: Change tailscale trigger condition (#30471)
younesbelkada Apr 25, 2024
e9b1635
FIX / Workflow: Fix SSH workflow bug (#30474)
younesbelkada Apr 25, 2024
c60749d
[fix codellama conversion] (#30472)
ArthurZucker Apr 25, 2024
30ee508
Script for finding candidate models for deprecation (#29686)
amyeroberts Apr 25, 2024
fac7bb3
Merge branch 'main' into gguf-support
younesbelkada Apr 25, 2024
4fed29e
Fix SigLip classification doctest (#30475)
amyeroberts Apr 25, 2024
aca4a10
Don't run fp16 MusicGen tests on CPU (#30466)
amyeroberts Apr 25, 2024
ce5ae5a
Prevent crash with `WandbCallback` with third parties (#30477)
tomaarsen Apr 25, 2024
90cb55b
🚨 Add training compatibility for Musicgen-like models (#29802)
ylacombe Apr 25, 2024
7b1170b
Add WSD scheduler (#30231)
visheratin Apr 25, 2024
9283313
Fix Issue #29817 Video Classification Task Guide Using Undeclared Va…
manju-rangam Apr 25, 2024
86f2569
Make accelerate install non-torch dependent (#30463)
muellerzr Apr 25, 2024
ad697f1
Introduce Stateful Callbacks (#29666)
muellerzr Apr 25, 2024
e60491a
Fix Llava for 0-embeddings (#30473)
zucchini-nlp Apr 25, 2024
bc274a2
Do not use deprecated `SourceFileLoader.load_module()` in dynamic mod…
XuehaiPan Apr 25, 2024
f396271
Add sidebar tutorial for chat models (#30401)
Rocketknight1 Apr 25, 2024
26ddc58
Quantization: `HfQuantizer` quant method update (#30484)
younesbelkada Apr 25, 2024
a98c417
[docs] Spanish translation of pipeline_tutorial.md (#30252)
aaronjimv Apr 25, 2024
19cfdf0
FEAT: PEFT support for EETQ (#30449)
younesbelkada Apr 26, 2024
59e715f
Fix the `bitsandbytes` error formatting ("Some modules are dispatched…
kyo-takano Apr 26, 2024
20081c7
Update `dtype_byte_size` to handle torch.float8_e4m3fn/float8_e5m2 ty…
mgoin Apr 26, 2024
2de5cb1
Use the Keras set_random_seed in tests (#30504)
Rocketknight1 Apr 26, 2024
dfa7b58
[`BERT`] Add support for sdpa (#28802)
hackyon Apr 26, 2024
77ff304
Remove skipping logic now that set_epoch exists (#30501)
muellerzr Apr 26, 2024
aafa7ce
[`DETR`] Remove timm hardcoded logic in modeling files (#29038)
amyeroberts Apr 26, 2024
38b53da
[examples] update whisper fine-tuning (#29938)
sanchit-gandhi Apr 26, 2024
e7d52a1
Fix GroundingDINO, DPR after BERT SDPA update (#30506)
amyeroberts Apr 26, 2024
c793b26
load_image - decode b64encode and encodebytes strings (#30192)
amyeroberts Apr 26, 2024
6d4cabd
[SegGPT] Fix seggpt image processor (#29550)
EduardoPach Apr 26, 2024
73014b5
Fix link in dbrx.md (#30509)
eitanturok Apr 26, 2024
80126f9
Allow boolean FSDP options in fsdp_config (#30439)
helloworld1 Apr 29, 2024
e8acb70
Pass attn_implementation when using AutoXXX.from_config (#30507)
amyeroberts Apr 29, 2024
bdbe166
Fix broken link to Transformers notebooks (#30512)
clinty Apr 29, 2024
c024218
Update runner tag for PR slow CI (#30535)
ydshieh Apr 29, 2024
87be06c
Fix repo. fetch/checkout in PR slow CI job (#30537)
ydshieh Apr 29, 2024
9df8b30
Reenable SDPA's FA2 During Training with torch.compile (#30442)
warner-benjamin Apr 29, 2024
a3aabc7
Include safetensors as part of `_load_best_model` (#30553)
muellerzr Apr 29, 2024
c712d05
Pass `use_cache` in kwargs for GPTNeoX (#30538)
zucchini-nlp Apr 30, 2024
0ae789e
Enable multi-device for more models (#30409)
jla524 Apr 30, 2024
45983db
Merge remote-tracking branch 'upstream/main' into HEAD
younesbelkada Apr 30, 2024
e6c6f6c
change logic
younesbelkada Apr 30, 2024
a6cd08c
add comment for clarity
younesbelkada Apr 30, 2024
6611877
add comment for clarity
younesbelkada Apr 30, 2024
455163b
Update src/transformers/models/auto/tokenization_auto.py
younesbelkada Apr 30, 2024
42d5815
change logic
younesbelkada Apr 30, 2024
1d3acec
Update src/transformers/modeling_utils.py
younesbelkada Apr 30, 2024
af3c42c
change
younesbelkada Apr 30, 2024
a27db0c
Merge branch 'gguf-support' of https://github.com/lysandrejik/transfo…
younesbelkada Apr 30, 2024
14ad10c
Apply suggestions from code review
younesbelkada Apr 30, 2024
ab621a7
Update src/transformers/modeling_gguf_pytorch_utils.py
younesbelkada Apr 30, 2024
207820a
put back comment
younesbelkada Apr 30, 2024
1fef8ad
add comment about mistral
younesbelkada Apr 30, 2024
9ae7363
comments and added tests
younesbelkada Apr 30, 2024
75bbfd5
Cache: Static cache as a standalone object (#30476)
gante Apr 30, 2024
1bff6a0
Generate: update links on LLM tutorial doc (#30550)
gante Apr 30, 2024
78a57c5
DBRX: make fixup (#30578)
gante Apr 30, 2024
9112520
Fix seq2seq collator padding (#30556)
vasqu Apr 30, 2024
0cdb6b3
BlipModel: get_multimodal_features method (#30438)
XavierSpycy Apr 30, 2024
2ecefc3
Add chat templating support for KeyDataset in text-generation pipelin…
DarshanDeshpande Apr 30, 2024
b8ac4d0
Fix generation doctests (#30263)
zucchini-nlp Apr 30, 2024
87927b2
General PR slow CI (#30540)
ydshieh Apr 30, 2024
78fdd64
Remove `use_square_size` after loading (#30567)
ydshieh Apr 30, 2024
9d31b32
Use text config's vocab size in testing models (#30568)
zucchini-nlp May 1, 2024
38a4bf7
Encoder-decoder models: move embedding scale to nn.Module (#30410)
zucchini-nlp May 1, 2024
4bc9cb3
Fix Marian model conversion (#30173)
zucchini-nlp May 1, 2024
4b4da18
Refactor default chat template warnings (#30551)
Rocketknight1 May 1, 2024
1e05671
Fix QA example (#30580)
Rocketknight1 May 1, 2024
3c69d81
remove jax example (#30498)
ArthurZucker May 1, 2024
bbaa8ce
Fix canonical model --model_type in examples (#30480)
amyeroberts May 1, 2024
f4f18af
Gemma: update activation warning (#29995)
pcuenca May 1, 2024
b71f512
Bump gitpython from 3.1.32 to 3.1.41 in /examples/research_projects/d…
dependabot[bot] May 1, 2024
ff8f624
Bump grpcio from 1.44.0 to 1.53.2 in /examples/research_projects/deci…
dependabot[bot] May 1, 2024
7164171
Bump gitpython from 3.1.32 to 3.1.41 in /examples/research_projects/d…
dependabot[bot] May 1, 2024
aefbdfe
Bump pyarrow from 7.0.0 to 15.0.0 in /examples/research_projects/deci…
dependabot[bot] May 1, 2024
0ec3003
Bump pyarrow from 1.0.1 to 15.0.0 in /examples/research_projects/visu…
dependabot[bot] May 1, 2024
b33f01f
Bump pyarrow from 1.0.1 to 15.0.0 in /examples/research_projects/lxme…
dependabot[bot] May 1, 2024
7a29c57
Bump torch from 1.6.0 to 1.13.1 in /examples/research_projects/lxmert…
dependabot[bot] May 1, 2024
ce66c0e
Bump torch from 1.11.0 to 1.13.1 in /examples/research_projects/codep…
dependabot[bot] May 1, 2024
6e0cba3
Bump torch from 1.6.0 to 1.13.1 in /examples/research_projects/visual…
dependabot[bot] May 1, 2024
d2feb54
Fix image segmentation example - don't reopen image (#30481)
amyeroberts May 1, 2024
dc401d3
Improve object detection task guideline (#29967)
NielsRogge May 1, 2024
d57ffb4
Generate: remove deprecated public decoding functions and streamline …
gante May 1, 2024
5090ea3
Fix llava half precision and autocast issues (#29721)
frasermince May 1, 2024
6f465d4
Bump torch from 1.11.0 to 1.13.1 in /examples/research_projects/decis…
dependabot[bot] May 1, 2024
4f3c7af
Bump torch from 1.9.0+cpu to 1.13.1 in /examples/research_projects/ja…
dependabot[bot] May 1, 2024
3a36597
Bump pillow from 10.0.1 to 10.2.0 in /examples/research_projects/deci…
dependabot[bot] May 1, 2024
c681b58
Bump torch from 1.9.0+cpu to 1.13.1 in /examples/flax/vision (#21168)
dependabot[bot] May 1, 2024
5cf3e6b
Fix: failing CI after #30568 (#30599)
zucchini-nlp May 2, 2024
fbabd67
Fix for Neuron (#30259)
michaelbenayoun May 2, 2024
12c5544
Fix memory leak with CTC training script on Chinese languages (#30358)
lucky-bai May 2, 2024
f953025
🚨 Update image_processing_vitmatte.py (#30566)
rb-synth May 2, 2024
4ad5ada
Fix copies for DBRX - neuron fix (#30610)
amyeroberts May 2, 2024
a65da83
fix:missing `output_router_logits` in SwitchTransformers (#30573)
lausannel May 2, 2024
f57f014
Use `contiguous()` in clip checkpoint conversion script (#30613)
ydshieh May 2, 2024
801894e
phi3 chat_template does not support system role (#30606)
amitportnoy May 2, 2024
aa55ff4
Docs: fix `generate`-related rendering issues (#30600)
gante May 2, 2024
66abe13
Docs: add missing `StoppingCriteria` autodocs (#30617)
gante May 2, 2024
9719202
Generate: fix `SinkCache` on Llama models (#30581)
gante May 2, 2024
39359e5
Fix FX tracing issues for Llama (#30619)
michaelbenayoun May 2, 2024
4c94093
Output `None` as attention when layer is skipped (#30597)
jonghwanhyeon May 2, 2024
5995299
Add HQQ quantization support (#29637)
mobicham May 2, 2024
a0e77a1
Fix CI after #30410 (#30612)
zucchini-nlp May 2, 2024
425e1a0
add mlp bias for llama models (#30031)
mayank31398 May 3, 2024
66f675e
Fix W&B run name (#30462)
qubvel May 3, 2024
d0c72c1
HQQ: PEFT support for HQQ (#30632)
younesbelkada May 3, 2024
deb7605
Prevent `TextGenerationPipeline._sanitize_parameters` from overriding…
yting27 May 3, 2024
91d155e
Avoid duplication in PR slow CI model list (#30634)
ydshieh May 3, 2024
307f632
[`CI update`] Try to use dockers and no cache (#29202)
ArthurZucker May 6, 2024
09edd77
Check if the current compiled version of pytorch supports MPS (#30664)
jiaqianjing May 6, 2024
a45c514
Hotfix-change-ci (#30669)
ArthurZucker May 6, 2024
9c772ac
Quantization / HQQ: Fix HQQ tests on our runner (#30668)
younesbelkada May 6, 2024
aa64f08
Fix llava next tie_word_embeddings config (#30640)
SunMarc May 6, 2024
e076953
Trainer._load_from_checkpoint - support loading multiple Peft adapter…
claralp May 6, 2024
df475bf
Trainer - add cache clearing and the option for batched eval metrics …
FoamoftheSea May 6, 2024
277db23
Fix typo: llama3.md (#30653)
mimbres May 6, 2024
835de4c
Respect `resume_download` deprecation (#30620)
Wauplin May 6, 2024
4980d62
top-k instead of top-p in MixtralConfig docstring (#30687)
sorgfresser May 7, 2024
a898fb9
Bump jinja2 from 3.1.3 to 3.1.4 in /examples/research_projects/decisi…
dependabot[bot] May 7, 2024
ce47582
Bump werkzeug from 3.0.1 to 3.0.3 in /examples/research_projects/deci…
dependabot[bot] May 7, 2024
54a2361
Adding _tie_weights() to prediction heads to support low_cpu_mem_usag…
hackyon May 7, 2024
4fda78c
Fix `cache_position` initialisation for generation with `use_cache=Fa…
nurlanov-zh May 7, 2024
9c8979e
Word-level timestamps broken for short-form audio (#30325)
kamilakesbi May 7, 2024
e5f71ec
Updated docs of `forward` in `Idefics2ForConditionalGeneration` with …
zafstojano May 7, 2024
4051d36
Bump tqdm from 4.63.0 to 4.66.3 in /examples/research_projects/decisi…
dependabot[bot] May 7, 2024
3733391
Bump tqdm from 4.48.2 to 4.66.3 in /examples/research_projects/visual…
dependabot[bot] May 7, 2024
0ba15ce
Reboot Agents (#30387)
aymeric-roucher May 7, 2024
4a17200
Bump tqdm from 4.48.2 to 4.66.3 in /examples/research_projects/lxmert…
dependabot[bot] May 7, 2024
4208c42
Separate tokenizer tests (#30675)
ArthurZucker May 7, 2024
05ec950
Update `workflow_id` in `utils/get_previous_daily_ci.py` (#30695)
ydshieh May 7, 2024
884e3b1
Rename artifact name `prev_ci_results` to `ci_results` (#30697)
ydshieh May 7, 2024
cf7bed9
Add safetensors to model not found error msg for default use_safetens…
davidgxue May 7, 2024
5b7a225
Pin deepspeed (#30701)
muellerzr May 7, 2024
508c0bf
Patch CLIP image preprocessor (#30698)
rootonchair May 8, 2024
998dbe0
Add examples for detection models finetuning (#30422)
qubvel May 8, 2024
1872bde
[BitsandBytes] Verify if GPU is available (#30533)
NielsRogge May 8, 2024
467164e
Llava: remove dummy labels (#30706)
zucchini-nlp May 8, 2024
e7a5f45
Add installation of examples requirements in CI (#30708)
qubvel May 8, 2024
5962d62
Update object detection guide (#30683)
qubvel May 8, 2024
71c1985
Immutability for data collators (#30603)
vasqu May 8, 2024
f26e407
Cache: models return input cache type (#30716)
gante May 8, 2024
60293bd
Add dynamic resolution input/interpolate position embedding to SigLIP…
davidgxue May 9, 2024
8c5b3c1
Enable dynamic resolution for vivit (#30630)
jla524 May 9, 2024
297b732
Removal of deprecated maps (#30576)
LysandreJik May 9, 2024
df53c6e
Generate: add `min_p` sampling (#30639)
gante May 9, 2024
218f441
Fix image post-processing for OWLv2 (#30686)
jla524 May 9, 2024
5413b89
KV cache is no longer a model attribute (#30730)
zucchini-nlp May 9, 2024
7130a22
Generate: consistently handle special tokens as tensors (#30624)
gante May 9, 2024
c99d88e
Update CodeLlama references (#30218)
osanseviero May 9, 2024
47735f5
[docs] Update es/pipeline_tutorial.md (#30684)
aaronjimv May 9, 2024
73fcfb2
Update llama3.md, fix typo (#30739)
mimbres May 10, 2024
1c52cb7
mlp_only_layers is more flexible than decoder_sparse_step (#30552)
eigen2017 May 10, 2024
2d1602a
PEFT / Trainer: Make use of `model.active_adapters()` instead of depr…
younesbelkada May 10, 2024
8ce4fef
[docs] Update link in es/pipeline_webserver.md (#30745)
aaronjimv May 10, 2024
e0c3cee
hqq - fix weight check in check_quantized_param (#30748)
mobicham May 10, 2024
de6e0db
[awq] replace scale when we have GELU (#30074)
SunMarc May 13, 2024
a4e530e
Workflow: Replace `actions/post-slack` with centrally defined workflo…
younesbelkada May 13, 2024
f63d822
Blip dynamic input resolution (#30722)
zafstojano May 13, 2024
e52741f
Support for Falcon2-11B (#30771)
Nilabhra May 13, 2024
453893e
[GroundingDino] Adding ms_deform_attn kernels (#30768)
EduardoPach May 13, 2024
a0779b9
Llama: fix custom 4D masks, v2 (#30348)
poedator May 13, 2024
f823fec
Generation / FIX: Fix multi-device generation (#30746)
younesbelkada May 13, 2024
f4dc26d
Qwen: incorrect setup flag (#30776)
gante May 13, 2024
69d9bca
enable Pipeline to get device from model (#30534)
faaany May 13, 2024
ce87dca
[Object detection pipeline] Lower threshold (#30710)
NielsRogge May 13, 2024
de2f722
Generate: remove near-duplicate sample/greedy copy (#30773)
gante May 13, 2024
9430635
Port IDEFICS to tensorflow (#26870)
a8nova May 13, 2024
2e27291
Generate: assistant should be greedy in assisted decoding (#30778)
gante May 13, 2024
82c1625
Save other CI jobs' result (torch/tf pipeline, example, deepspeed etc…
ydshieh May 13, 2024
0f8fefd
Deprecate models script (#30184)
amyeroberts May 13, 2024
539ed75
skip low_cpu_mem_usage tests (#30782)
SunMarc May 13, 2024
37bba2a
CI: update to ROCm 6.0.2 and test MI300 (#30266)
fxmarty May 13, 2024
449894d
Fix OWLv2 Doc (#30794)
jla524 May 14, 2024
c02d302
Fix cache type in Idefics2 (#30729)
zucchini-nlp May 14, 2024
65ea190
PEFT: Access active_adapters as a property in Trainer (#30790)
pashminacameron May 14, 2024
5ad960f
Add Watermarking LogitsProcessor and WatermarkDetector (#29676)
zucchini-nlp May 14, 2024
3ed384f
fix merge
younesbelkada May 14, 2024
55eb860
fix unconsistent type
younesbelkada May 14, 2024
f754335
more
younesbelkada May 14, 2024
d8f8a9c
CI: more models wo cache support (#30780)
gante May 14, 2024
a449078
Merge remote-tracking branch 'origin/main' into HEAD
younesbelkada May 14, 2024
3bdbb2e
fix tokenizer
younesbelkada May 15, 2024
0ab79f6
Update src/transformers/modeling_utils.py
younesbelkada May 15, 2024
65433c4
address comments about tests and tokenizer + add added_tokens
younesbelkada May 15, 2024
1b5ae54
from_gguf -> gguf_file
younesbelkada May 15, 2024
d6b67c6
replace on docs too
younesbelkada May 15, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
64 changes: 12 additions & 52 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ jobs:
# Ensure running with CircleCI/huggingface
check_circleci_user:
docker:
- image: cimg/python:3.8.12
- image: python:3.10-slim
parallelism: 1
steps:
- run: echo $CIRCLE_PROJECT_USERNAME
Expand All @@ -26,13 +26,11 @@ jobs:
fetch_tests:
working_directory: ~/transformers
docker:
- image: cimg/python:3.8.12
- image: huggingface/transformers-quality
parallelism: 1
steps:
- checkout
- run: pip install --upgrade --upgrade-strategy eager pip
- run: pip install -U --upgrade-strategy eager GitPython
- run: pip install -U --upgrade-strategy eager .
- run: uv pip install -U -e .
- run: mkdir -p test_preparation
- run: python utils/tests_fetcher.py | tee tests_fetched_summary.txt
- store_artifacts:
Expand Down Expand Up @@ -88,25 +86,22 @@ jobs:
echo "No tests to run, exiting early!"
circleci-agent step halt
fi
- run: cp test_preparation/generated_config.yml test_preparation/generated_config.txt
- store_artifacts:
path: test_preparation/generated_config.txt
path: test_preparation/generated_config.yml
- store_artifacts:
path: test_preparation/filtered_test_list_cross_tests.txt
path: test_preparation/filtered_test_list_cross_tests.txt
- continuation/continue:
configuration_path: test_preparation/generated_config.yml
configuration_path: test_preparation/generated_config.yml

# To run all tests for the nightly build
fetch_all_tests:
working_directory: ~/transformers
docker:
- image: cimg/python:3.8.12
- image: huggingface/transformers-consistency
parallelism: 1
steps:
- checkout
- run: pip install --upgrade --upgrade-strategy eager pip
- run: pip install -U --upgrade-strategy eager GitPython
- run: pip install -U --upgrade-strategy eager .
- run: uv pip install -e .
- run: |
mkdir test_preparation
echo -n "tests" > test_preparation/test_list.txt
Expand All @@ -126,32 +121,15 @@ jobs:
check_code_quality:
working_directory: ~/transformers
docker:
- image: cimg/python:3.8.12
- image: huggingface/transformers-quality
resource_class: large
environment:
TRANSFORMERS_IS_CI: yes
PYTEST_TIMEOUT: 120
parallelism: 1
steps:
- checkout
- restore_cache:
keys:
- v0.7-code_quality-pip-{{ checksum "setup.py" }}
- v0.7-code-quality-pip
- restore_cache:
keys:
- v0.7-code_quality-site-packages-{{ checksum "setup.py" }}
- v0.7-code-quality-site-packages
- run: pip install --upgrade --upgrade-strategy eager pip
- run: pip install -U --upgrade-strategy eager .[all,quality]
- save_cache:
key: v0.7-code_quality-pip-{{ checksum "setup.py" }}
paths:
- '~/.cache/pip'
- save_cache:
key: v0.7-code_quality-site-packages-{{ checksum "setup.py" }}
paths:
- '~/.pyenv/versions/'
- run: uv pip install -e .
- run:
name: Show installed libraries and their versions
command: pip freeze | tee installed.txt
Expand All @@ -167,32 +145,15 @@ jobs:
check_repository_consistency:
working_directory: ~/transformers
docker:
- image: cimg/python:3.8.12
- image: huggingface/transformers-consistency
resource_class: large
environment:
TRANSFORMERS_IS_CI: yes
PYTEST_TIMEOUT: 120
parallelism: 1
steps:
- checkout
- restore_cache:
keys:
- v0.7-repository_consistency-pip-{{ checksum "setup.py" }}
- v0.7-repository_consistency-pip
- restore_cache:
keys:
- v0.7-repository_consistency-site-packages-{{ checksum "setup.py" }}
- v0.7-repository_consistency-site-packages
- run: pip install --upgrade --upgrade-strategy eager pip
- run: pip install -U --upgrade-strategy eager .[all,quality]
- save_cache:
key: v0.7-repository_consistency-pip-{{ checksum "setup.py" }}
paths:
- '~/.cache/pip'
- save_cache:
key: v0.7-repository_consistency-site-packages-{{ checksum "setup.py" }}
paths:
- '~/.pyenv/versions/'
- run: uv pip install -e .
- run:
name: Show installed libraries and their versions
command: pip freeze | tee installed.txt
Expand All @@ -208,7 +169,6 @@ jobs:
- run: python utils/check_doctest_list.py
- run: make deps_table_check_updated
- run: python utils/update_metadata.py --check-only
- run: python utils/check_task_guides.py
- run: python utils/check_docstrings.py
- run: python utils/check_support_list.py

Expand Down
Loading
Loading