CUDA is not working

[LogosnetDev]

Hello there!
I was trying the plugin running on cuda 12.2

i think it is not properly working.
here steps i did

1. import package from git url on my project
2. created the script to manage the whisper manager and the mic record (according to the sample)
3. tested "one, two, three test" on cpu (it is working fine)
   
4. enabled the CUDA toggle and restarting unity project.
5. the testing result this one on recorded sentece "one two three test", it took a lot of time and the result is unexpected
   

specs
OS Name: Microsoft Windows 10 Pro
OS Version: 10.0.19045 N/A Build 19045
GPU NVIDIA 1060 ti
CUDA 12.2
RAM 16 gb
CPU Intel64_Family_6_Model_158_-_Intel(R)_Core(TM)i5-7600K_CPU@_3.80GHz_1

[Macoron]

Thanks for reporting! Seems like whisper.cpp bug. Could you try to run original whisper.cpp release compiled with cuBLAS to see if it works for you?

https://github.com/ggerganov/whisper.cpp/releases/tag/v1.5.1

[LogosnetDev]

yeah it seems to be a source problem.
result using no gpu option

result using the gpu

so i think i should move this report to the original repo.

I noticed this executable running much more faster then the one running on the unity plugin. the clip is elaborated in around 2 seconds, but in unity package it takes around 20 secs. am i missing something?
thank you for the supporting

[Macoron]

I noticed this executable running much more faster then the one running on the unity plugin. the clip is elaborated in around 2 seconds, but in unity package it takes around 20 secs. am i missing something?

Hard to say for sure. Unity has some overhead, but not 20x or something. On my RTX 2070 Super similar length audio takes around 100 ms. I think the problem in some whisper.cpp bug for your GPU.

Could you also quickly test 1.5.4 or 1.5.5 releases of whisper.cpp? whisper.unity uses 1.5.1 version, so maybe your problem was fixed in latter releases.

[LogosnetDev]

thank you for you help, i can confirm this bug is fixed in the 1.5.4 release

[LogosnetDev]

there any way to replace the plugin used one one with the newer releses?

[Macoron]

I updated whisper.cpp to 1.5.5 in #84. Try to download latest master and check if it fixes issue.

[Gabri94x]

@Macoron hello there, thank you for you update, i can confirm you now it works properly.
thank you!