6700XT only works up to KoboldCPP-v1.76.yr0-ROCm

[o0LINNY0o]

6700XT only works up to KoboldCPP-v1.76.yr0-ROCm, anything beyond that and i get the following error:

ROCm error: CUBLAS_STATUS_INTERNAL_ERROR
current device: 0, in function ggml_cuda_mul_mat_batched_cublas at D:/a/koboldcpp-rocm/koboldcpp-rocm/ggml/src/ggml-cuda.cu:1881
hipblasGemmBatchedEx(ctx.cublas_handle(), HIPBLAS_OP_T, HIPBLAS_OP_N, ne01, ne11, ne10, alpha, (const void **) (ptrs_src.get() + 0ne23), HIPBLAS_R_16F, nb01/nb00, (const void **) (ptrs_src.get() + 1ne23), HIPBLAS_R_16F, nb11/nb10, beta, ( void **) (ptrs_dst.get() + 0*ne23), cu_data_type, ne01, ne23, cu_compute_type, HIPBLAS_GEMM_DEFAULT)
D:/a/koboldcpp-rocm/koboldcpp-rocm/ggml/src/ggml-cuda.cu:72: ROCm error

[YellowRoseCx]

So KoboldCPP-v1.76.yr0 works for you, and KoboldCPP-v1.76.yr1 does not?

[o0LINNY0o]

Correct , great project btw.

[lanifn]

Can confirm that this is not just isolated to o0LINNY0o's system.

[GramSeco]

I have a 6750 XT, same issue. I updated from 1.72 to the latest version and ran into issues. Any version beyond 1.76.yr0 does not work with hipBLAS (ROCm). Vulkan does work on all versions.

[PDCWolf]

Owner of a 6750XT, can confirm this is exactly the case too. Same error.

[YellowRoseCx]

I'll look into it more this evening but it doesn't really make sense to me why there's an issue because the only change I made from 1.76 to 1.76.yr1 was using the official ROCM GPU files 🥲🕵️🖥️ and they provide gfx1031 files.. I guess the ones I compiled had a magic touch in them 😜

[YellowRoseCx]

let me know if this version works for you guys https://github.com/YellowRoseCx/koboldcpp-rocm/releases/tag/v1.79.1.yr0-ROCm

[YellowRoseCx]

Do you have any luck with this version?

https://github.com/YellowRoseCx/koboldcpp-rocm/releases/tag/v1.79.1.yr1-ROCm

Its different than what I posted 2 days ago

[o0LINNY0o]

my bad, i missed that , yes TY its working. What has the change

[YellowRoseCx]

I changed the GPU rocblas library files and the rocblas.dll from the one that comes in the ROCm install with the ones I compiled; no clue why they work and the ROCm ones didn't! I just hope it still works for the other GPUs lol

[legodark-hub]

I changed the GPU rocblas library files and the rocblas.dll from the one that comes in the ROCm install with the ones I compiled; no clue why they work and the ROCm ones didn't! I just hope it still works for the other GPUs lol

Looks like the official library has limited support for 6600-6750 XT: https://rocm.docs.amd.com/projects/install-on-windows/en/latest/reference/system-requirements.html#windows-supported-gpus

[GramSeco]

Just tried this version and it works. Thank you

[o0LINNY0o]

TY - but still getting the error [6700XT]

ROCm error: CUBLAS_STATUS_INTERNAL_ERROR
current device: 0, in function ggml_cuda_mul_mat_batched_cublas at D:/a/koboldcpp-rocm/koboldcpp-rocm/ggml/src/ggml-cuda/ggml-cuda.cu:1718
hipblasGemmBatchedEx(ctx.cublas_handle(), HIPBLAS_OP_T, HIPBLAS_OP_N, ne01, ne11, ne10, alpha, (const void **) (ptrs_src.get() + 0ne23), HIPBLAS_R_16F, nb01/nb00, (const void **) (ptrs_src.get() + 1ne23), HIPBLAS_R_16F, nb11/nb10, beta, ( void **) (ptrs_dst.get() + 0*ne23), cu_data_type, ne01, ne23, cu_compute_type, HIPBLAS_GEMM_DEFAULT)
D:/a/koboldcpp-rocm/koboldcpp-rocm/ggml/src/ggml-cuda/ggml-cuda.cu:72: ROCm error

[YellowRoseCx]

Which model and what settings are you using?

[o0LINNY0o]

EDIT: For testing I just use the llama3.1 8b .

Basic default settings. 4k context 512 batch size.

also using Drivers Adrenaline 24.10.1. do you think its worth rolling back ?? if so what version

[legodark-hub]

I'm on 6600xt and I get the same error with hipBLAS since v1.76.yr1, on models that worked on previous versions.
I ran everything on default settings.
The latest updates didn't help.
screenshot from the latest version.

[YellowRoseCx]

Is that with this version? https://github.com/YellowRoseCx/koboldcpp-rocm/releases/tag/v1.79.1.yr1-ROCm

[legodark-hub]

oh, no, my screenshot was from this version https://github.com/YellowRoseCx/koboldcpp-rocm/releases/tag/v1.79.1.yr0-ROCm
but in the version you specified, everything works👍

[LinkCodes]

Same problem whit a 6600XT since the 1.74 version, download the 1.79.1yr1 its working now, Thank you. awesome work!!

[GramSeco]

I hate to necro this, but I think we might be back to the same issue with v 1.80.3.yr0. It closes almost immediately upon first message sent.

[TheLapinMalin]

Same issue starting from 1.80.3:

ROCm error: CUBLAS_STATUS_INTERNAL_ERROR current device: 0, in function ggml_cuda_mul_mat_batched_cublas at D:/a/koboldcpp-rocm/koboldcpp-rocm/ggml/src/ggml-cuda/ggml-cuda.cu:1719 hipblasGemmBatchedEx(ctx.cublas_handle(), HIPBLAS_OP_T, HIPBLAS_OP_N, ne01, ne11, ne10, alpha, (const void **) (ptrs_src.get() + 0*ne23), HIPBLAS_R_16F, nb01/nb00, (const void **) (ptrs_src.get() + 1*ne23), HIPBLAS_R_16F, nb11/nb10, beta, ( void **) (ptrs_dst.get() + 0*ne23), cu_data_type, ne01, ne23, cu_compute_type, HIPBLAS_GEMM_DEFAULT) D:/a/koboldcpp-rocm/koboldcpp-rocm/ggml/src/ggml-cuda/ggml-cuda.cu:73: ROCm error

[antonio-RN]

Issues also here with 6750XT. Up until 1.79.1 it's working fine, but from 1.80 on it gives different errors.

• 1.80.3.yr0-ROCm & 1.81.1.yr0-ROCm (different issue?)

  Job 1, 'python ./koboldcpp.py' terminated by signal SIGSEGV (Address boundary error)

• 1.82.yr0-ROCm and beyond (hipblas issue)

  Traceback (most recent call last):
    File "/home/antonior/koboldcpp-rocm/./koboldcpp.py", line 6004, in <module>
      main(parser.parse_args(),start_server=True)
    File "/home/antonior/koboldcpp-rocm/./koboldcpp.py", line 5482, in main
      init_library() # Note: if blas does not exist and is enabled, program will crash.
      ^^^^^^^^^^^^^^
    File "/home/antonior/koboldcpp-rocm/./koboldcpp.py", line 491, in init_library
      handle.tts_load_model.argtypes = [tts_load_model_inputs]
      ^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib/python3.12/ctypes/__init__.py", line 392, in __getattr__
      func = self.__getitem__(name)
             ^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib/python3.12/ctypes/__init__.py", line 397, in __getitem__
      func = self._FuncPtr((name_or_ordinal, self))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  AttributeError: /home/antonior/koboldcpp-rocm/koboldcpp_hipblas.so: undefined symbol: tts_load_model. Did you mean: 'sd_load_model'?
  

Using Python 3.12.3 with a Ryzen 5600x and AMD 6750XT.

ROCk module is loaded
=====================    
HSA System Attributes    
=====================    
Runtime Version:         1.14
Runtime Ext Version:     1.6

[VeleSila]

@YellowRoseCx

[YellowRoseCx]

I'll see if I can reproduce it when I get home