auto-detect device when no device is passed to pipeline

[faaany]

What does this PR do?

Currently, if no device is passed to pipeline, the model will stay on CPU. This PR makes it possible to auto-detect the underlying hardware environment and move the model to the corresponding accelerator.

import torch 
from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline

model_id = "meta-llama/Llama-2-7b-chat-hf"
tokenizer = AutoTokenizer.from_pretrained(model_id)
 
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float16)
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer, do_sample=False, max_new_tokens=300, num_beams=1)
print(pipe.model.device)
### before fix: cpu
### after fix: cuda

@amyeroberts and @muellerzr

[amyeroberts]

Thanks for adding this @faaany

I'm not sure this is something we want to do - it can both lead to unexpected behaviour for the user (the default is changing), and doesn't match with the rest of the library (we don't automatically put a model onto a GPU if it's available in the environment when instantiating it). WDYT @ArthurZucker?

[faaany]

@yao-matrix

[faaany]

Hi @ArthurZucker , any thoughts on this PR?

[muellerzr]

0.02$, also don't think we should do this automatically for the user. Feels a bit too "magical", even if it's useful. I'd rather switch to a good default to use the device if possible instead so it's exposed at the top level, if we were to.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[amyeroberts]

How about we emit a warning if we detect there's a device but none is set for the pipeline?

[muellerzr]

A warning sounds good, and at this level I feel users would pay attention to it (rather than it just being bloat on the warning logs etc)

[faaany]

Thx for the suggestion! I will update the PR.

[faaany]

I think we need to re-trigger the CI

[amyeroberts]

@faaany Could you try rebasing to include upstream changes from main?

[faaany]

@faaany Could you try rebasing to include upstream changes from main?

sure, done.

[amyeroberts]

Thanks for adding!