AttributeError: 'TrainingArguments' object has no attribute 'packing' #qlora, STFTrainer #unsloth

[gulsumbudakoglu]

System Info

I use the SFTTrainer for my qlora fine-tuning for Mistral Instruct 2 model. I use unsloth to make my training faster. I have run the code multiple times before but today I got the AttributeError: 'TrainingArguments' object has no attribute 'packing' error immediatly.
Are there anyone who got the same error and know the solution?

Thank a lot!

I run my code on colab -T4
I have used the following libraries:

!pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
!pip install --no-deps xformers trl peft accelerate bitsandbytes

my error explanation also:

AttributeError                            Traceback (most recent call last)
[<ipython-input-22-fee37d14dff4>](https://localhost:8080/#) in <cell line: 13>()
     11 
     12 
---> 13 trainer = SFTTrainer(
     14     model = model,
     15     tokenizer = tokenizer,

1 frames
[/usr/local/lib/python3.10/dist-packages/trl/trainer/sft_trainer.py](https://localhost:8080/#) in __init__(self, model, args, data_collator, train_dataset, eval_dataset, tokenizer, model_init, compute_metrics, callbacks, optimizers, preprocess_logits_for_metrics, peft_config, dataset_text_field, packing, formatting_func, max_seq_length, infinite, num_of_sequences, chars_per_token, dataset_num_proc, dataset_batch_size, neftune_noise_alpha, model_init_kwargs, dataset_kwargs, eval_packing)
    187             args.eval_packing = eval_packing
    188 
--> 189         if args.packing and data_collator is not None and isinstance(data_collator, DataCollatorForCompletionOnlyLM):
    190             raise ValueError(
    191                 "You passed a `DataCollatorForCompletionOnlyLM` to the SFTTrainer. This is not compatible with the `packing` argument."

AttributeError: 'TrainingArguments' object has no attribute 'packing'

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    train_dataset=train_data,
    dataset_text_field="text",
    max_seq_length=max_seq_length,
    dataset_num_proc=2,
    packing=False,  # Can make training 5x faster for short sequences.
    args=TrainingArguments(
        per_device_train_batch_size=2,
        gradient_accumulation_steps=4,
        warmup_steps=5,
        max_steps=60,  # Set num_train_epochs = 1 for full training runs
        learning_rate=2e-4,
        fp16=not is_bfloat16_supported(),
        bf16=is_bfloat16_supported(),
        logging_steps=1,
        optim="adamw_8bit",
        weight_decay=0.01,
        lr_scheduler_type="linear",
        seed=3407,
        save_steps=1,
        output_dir=output_dir,
    ),
)

Expected behavior

My code should run without error which is AttributeError: 'TrainingArguments' object has no attribute 'packing'

[amyeroberts]

cc @younesbelkada

[younesbelkada]

Hi
We recently made a TRL release and it seems to be not backward compatible with TrainingArguments. We advise users to use SFTConfig instead of TrainingArguments

+ from trl import SFTConfig

trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    train_dataset=train_data,
    dataset_text_field="text",
    max_seq_length=max_seq_length,
    dataset_num_proc=2,
    packing=False,  # Can make training 5x faster for short sequences.
-   args=TrainingArguments(
+   args=SFTConfig(
        per_device_train_batch_size=2,
        gradient_accumulation_steps=4,
        warmup_steps=5,
        max_steps=60,  # Set num_train_epochs = 1 for full training runs
        learning_rate=2e-4,
        fp16=not is_bfloat16_supported(),
        bf16=is_bfloat16_supported(),
        logging_steps=1,
        optim="adamw_8bit",
        weight_decay=0.01,
        lr_scheduler_type="linear",
        seed=3407,
        save_steps=1,
        output_dir=output_dir,
    ),
)

I will make sure SFTTrainer is going to be backward compatible with TrainingArguments and we will make a patch release soon
cc @vwxyzjn

[gulsumbudakoglu]

It worked with SFTConfig. Thank you for your support!

[Vincent-Li-9701]

This issue seems to appear again in version 0.9.6. Is it that we have decided to not be backward compatible on this?

cc. @younesbelkada

[amyeroberts]

Moving this to TRL as @younesbelkada is no longer with HF and they'll have the best knowledge of any changes or updates that have occurred

[amyeroberts]

Actually - seems like I can't move the issue there. Perhaps @lewtun would know?

[lewtun]

@Vincent-Li-9701 can you please share a reproducible example in e.g. a Google Colab?