How do I use my GPU instead of CPU for Whisper on an M2 Mac?

[mdjamilkashemporosh]

I am working with Whisper for speech-to-text transcription on an M2 Mac, and I'd like to utilize the GPU for faster processing instead of relying solely on the CPU. I'm aware that the M2 chip supports GPU acceleration through the Metal API, but I'm unsure how to configure whisper to leverage the GPU.

[glangford]

Try --device mps from the command line.

Some people use this derivative project, which is ported to Apple silicon
https://github.com/ggerganov/whisper.cpp

...or the Hugging Face implementation
#984 (comment)

More information on the MPS backend
https://pytorch.org/docs/stable/notes/mps.html