KoboldCPP-v1.75.2.yr0-ROCm

[sinni800]

For me, this still breaks, I never could get any version to work tbh.

ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon RX 6950 XT, compute capability 10.3, VMM: no
Traceback (most recent call last):
File "koboldcpp.py", line 4925, in
File "koboldcpp.py", line 4567, in main
File "koboldcpp.py", line 889, in load_model
OSError: exception: access violation reading 0x0000000000000000

This was using a Mistral NeMo model that works on the CUDA version, same computer, same model file.

My PC has a 2080ti and a 6950xt, so I can run both.

[YellowRoseCx]

For me, this still breaks, I never could get any version to work tbh.

ggml_cuda_init: found 1 ROCm devices: Device 0: AMD Radeon RX 6950 XT, compute capability 10.3, VMM: no Traceback (most recent call last): File "koboldcpp.py", line 4925, in File "koboldcpp.py", line 4567, in main File "koboldcpp.py", line 889, in load_model OSError: exception: access violation reading 0x0000000000000000

This was using a Mistral NeMo model that works on the CUDA version, same computer, same model file.

My PC has a 2080ti and a 6950xt, so I can run both.

Both @sinni800 and @Krxon ,
Can you please tell me some more with the following info?

Your Operating System:
OS Version (For Windows: Press Windows key + R, and type winver):
CPU:
Graphics Card(s) installed in your PC:
Link to the exact model you are using (including the specific quantization):

Lastly, could you open KoboldCPP, setup all the options you would normally use, and "save" the settings and upload the ".kcpps" file here?
If you are using it from command-line without GUI, the command you use to start the program would be appreciated.

It will help me look into the problem more, thanks!

[sinni800]

Windows 11 23H2, 22631.4169
Ryzen 5950x
Radeon 6950xt (primary), GeForce 2080ti (secondary, no screens, only running koboldcpp CUDA)
https://huggingface.co/bartowski/Mistral-Nemo-Instruct-2407-GGUF/blob/main/Mistral-Nemo-Instruct-2407-Q5_K_M.gguf

But various LLama3 models didn't work either, with the same error.

the Kcpps file (i renamed it to json for Github) works 100% in CUDA KoboldCPP, but the 2080ti only has 11gb ram, and I want to use the extra gigabytes of my 6950xt.

bla - Kopie.json

What's also funny, the GUI shows the 2080ti as the primary rocm gpu, but in the commandline logs after, it shows the 6950xt properly on all coutns

[JustAMerePleb]

I've the same error like you guys, but it just happen without any warning despite me using the latest version for at least at least a few hours. Then all versions of koboldcpp_rocm.exe stop working for me, no matter how far back I go while keep giving the same error:

ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon RX 6700 XT, compute capability 10.3, VMM: no
Traceback (most recent call last):
File "koboldcpp.py", line 4925, in
File "koboldcpp.py", line 4567, in main
File "koboldcpp.py", line 889, in load_model
OSError: exception: access violation reading 0x0000000000000000
[12928] Failed to execute script 'koboldcpp' due to unhandled exception!

But the 'access violation' didn't happen if I drag and drop the config file(.kcpps) onto the koboldcpp_rocm.exe and the program just run without popping up the GUI.
Maybe the windows update screw up something? Cause it only run if I provided the config file directly.

[YellowRoseCx]

Have you tried restarting your computer?

[JustAMerePleb]

Have you tried restarting your computer?

Multiple time but it still have the 'access violation reading', I even try a clean reinstall of the drivers. But only the drag+drop config file to run work for me.

[aiwaverse]

I would like to add that I have experienced the same exact reason, and this started happening yesterday, as the day before I used koboldcpp with no problems, I assume Windows 11 did something that broke the application.
Same error, but with a 7900 XTX, just installed the latest driver (2024/10/01) and it still happens, as mentioned, running this without the GUI by dragging the file on the .exe works with no problems.
I tested on a Windows 10 Machine, same files, and it runs, which is why I think it's a Windows update,.

ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon RX 7900 XTX, compute capability 11.0, VMM: no
Traceback (most recent call last):
File "koboldcpp.py", line 4925, in
File "koboldcpp.py", line 4567, in main
File "koboldcpp.py", line 889, in load_model
OSError: exception: access violation reading 0x0000000000000000
[20660] Failed to execute script 'koboldcpp' due to unhandled exception!

[sinni800]

I tried it now too and it works perfectly if I save a config file first, then run it directly with the config file.

Thanks!