Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Complete list of dependencies? #93

Open
Sazu-bit opened this issue Jan 11, 2025 · 0 comments
Open

Complete list of dependencies? #93

Sazu-bit opened this issue Jan 11, 2025 · 0 comments

Comments

@Sazu-bit
Copy link

Sazu-bit commented Jan 11, 2025

Describe the Issue
I believe I'm missing dependencies. If I just install koboldcpp-rocm as described on the readme, it doesn't work and I have to search for other related things to install (which suggests that the guide was not written based on a clean build vm).

The arch linux doesn't work (at least not for me) out of the box and I usually end up grabbing the git file instead but I'm currently stuck with an rocblas error stating that I don't have a tensile file for gfx806 but there is no clue on where to find such a file.

Additional Information:
I am using Artix (no SystemD basically).
Please provide as much relevant information about your setup as possible, such as the Operating System, CPU, GPU, KoboldCpp Version, and relevant logs (helpful to include the launch params from the terminal output, flags and crash logs)
20250110-001-TieFighter.txt

I have attached my settings file, I simply click launch after that and then I get...

koboldcpp
***
Welcome to KoboldCpp - Version 1.81.1.yr0-ROCm
For command line arguments, please refer to --help
***
Auto Selected Default Backend...

Initializing dynamic library: koboldcpp_hipblas.so
==========
Namespace(model='', model_param='/mnt/data/AI/llm/LLaMA2-13B-Tiefighter.Q5_K_M.gguf', port=5001, port_param=5001, host='', launch=False, config=None, threads=8, usecublas=['normal', '0', 'nommq'], usevulkan=None, useclblast=None, usecpu=False, contextsize=16384, gpulayers=37, tensor_split=None, checkforupdates=False, ropeconfig=[0.0, 10000.0], blasbatchsize=1024, blasthreads=8, lora=None, noshift=False, nofastforward=False, nommap=False, usemlock=False, noavx2=False, failsafe=False, debugmode=0, onready='', benchmark='stdout', prompt='', promptlimit=100, multiuser=1, multiplayer=False, websearch=False, remotetunnel=False, highpriority=False, foreground=False, preloadstory=None, quiet=False, ssl=None, nocertify=True, mmproj=None, draftmodel=None, draftamount=8, draftgpulayers=999, draftgpusplit=None, password=None, ignoremissing=False, chatcompletionsadapter=None, flashattention=False, quantkv=0, forceversion=0, smartcontext=False, unpack='', nomodel=False, moeexperts=-1, showgui=False, skiplauncher=False, hordemodelname='', hordeworkername='', hordekey='', hordemaxctx=0, hordegenlen=0, sdmodel='', sdthreads=3, sdclamped=0, sdt5xxl='', sdclipl='', sdclipg='', sdvae='', sdvaeauto=False, sdquant=False, sdlora='', sdloramult=1.0, whispermodel='', hordeconfig=None, sdconfig=None, noblas=False)
==========
Loading Text Model: /mnt/data/AI/llm/LLaMA2-13B-Tiefighter.Q5_K_M.gguf

The reported GGUF Arch is: llama
Arch Category: 0

---
Identified as GGUF model: (ver 6)
Attempting to Load...
---
Using automatic RoPE scaling for GGUF. If the model has custom RoPE settings, they'll be used directly instead!
It means that the RoPE values written above will be replaced by the RoPE values indicated after loading.
System Info: AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | AMX_INT8 = 0 | FMA = 1 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | RISCV_VECT = 0 | WASM_SIMD = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | 

rocBLAS error: Cannot read /opt/rocm/lib/rocblas/library/TensileLibrary.dat: No such file or directory for GPU arch : gfx803
 List of available TensileLibrary Files : 
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx1101.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx1102.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx1100.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx908.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx906.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx1012.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx941.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx942.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx940.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx900.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx1030.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx1010.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx90a.dat"
/usr/bin/koboldcpp: line 2:  7641 Aborted                 python /usr/share/koboldcpp/koboldcpp.py "$@"

Further information may be needed. This is an Radeon RX590 GPU, which I assume is pre-vega, but I don't know how to configure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant