Docker File improvements and Docker-compose File Ideas

[Neresco]

Good Evening,

i have created a Dockerfile to create a "sort of.." working Docker Container for Kobolcpp-rocm.
I have some issues i cannot solve at the moment and need input.
Also i would be glad if someone can share Ideas for a Docker compose file.

The Dockerfile:
https://github.com/Neresco/koboldcpp-rocm-dockerprepare/blob/main/Dockerfile
Build with docker buildx in the last trys.

I could not get it to work, when i use the compile commands in the dockerfile.
make clean and make LLAMA_HIPBLAS=1 LLAMA_VULKAN=1 -j4

make clean are not executed.
make LLAMA_HIPBLAS=1 LLAMA_VULKAN=1 -j4 are executed but when the Container is finished,
koboldcpp fails to start with an error.

When i compile after the container is finished kobold is usable.
At least until the Container is recreated/changed in portainer.

Also i was not able to make koboldcpp start with the start command.
"python koboldcpp.py --config /models/config.json"
Example for config.json.
https://github.com/Neresco/koboldcpp-rocm-dockerprepare/blob/main/config.json
The Container starts just with '/bin/bash'

Docker Compose creation i have not started yet and if someone has tips or insights i would be glad.

[Krisseck]

I made my own Dockerfiles here: https://github.com/Krisseck/ROCm-Docker-Scripts/

Building and debugging was annoying, the issue is that when building the container, the build process needs access to the GPU (probably to determine the target model). I couldn't find any code where you can pass the /dev/kfd device to the docker build process.

The issue can be circumvented by setting the GPU_TARGETS when building, that way it doesn't try to determine the model from the GPU.