Preparing dataloaders in two accelerator.prepare calls when deepspeed is enabled issue.

[lqtrung1998]

Hi, I recently encounter this:

## Deepspeed enabled
dataloader1 = accelerator.prepare(dataloader1) # correctly prepared 
dataloader2 = accelerator.prepare(dataloader2) #  not correctly prepared

accelerate/src/accelerate/accelerator.py

Line 1154 in 03754c1

if deepspeed_plugin.deepspeed_config["train_micro_batch_size_per_gpu"] == "auto":

Correct me if I'm wrong but ... It seems that when deepspeed is enabled, only in the first prepare that the dataloader is prepared ... after the first prepare, I guess the train_micro_batch_size_per_gpu will not be 'auto', thus, the original dataloader2 is returned.

Is it a bug or it is expected and this has been mentioned somewhere in the doc ?

Thank you,
Trung

[sgugger]

cc @pacman100

[pacman100]

Hello @lqtrung1998, please let us know if the above PR fixes the issue