docs: update fedora cuda guide for 12.8 release

[teihome]

In this pull request the cuda-fedora.md guide has been updated to use the latest release of CUDA 12.8 (previously 12.6), that was uploaded on 2025-01-17.

The new release uses the current version of Fedora 41 (previously 39).

This guide continues to use the Toolbox environment to allow easy installation on Silverblue or Workstation systems alike.

This pull request also updates the CUDA section of the build document to be more clear and descriptive for compiling for explicit compute compatibility targets.

[teihome]

Converted to a draft, as there are some compatibility issues with using the newer NVIDIA drivers in the toolbox over the host.

[teihome]

Okay, I have resolved the issue, the issue was that nvidia-driver-cuda-libs was installed in the guest even when /usr/lib64/libcuda.so.1 was supplied by the host.

When the guest libcuda was older, it would not be updated, so it matched the version of the host, but now with the guest having a newer version (570) of the libraries than the host (565), it would break.

It is fixed by never installing nvidia-driver-cuda-libs to the guest filesystem if the host is supplying CUDA.

[da2ce7]

With only minor modification it worked for the www.runpod.io config, using the docker image: registry.fedoraproject.org/fedora-toolbox:41

dnf distro-sync --assumeyes --quiet > /dev/null;
dnf install vim-default-editor -y --allowerasing  --assumeyes --quiet > /dev/null;
dnf install @c-development @development-tools cmake sshd  --assumeyes --quiet > /dev/null;
dnf config-manager addrepo --from-repofile=https://developer.download.nvidia.com/compute/cuda/repos/fedora41/x86_64/cuda-fedora41.repo;
dnf download --destdir=/tmp/nvidia-driver-libs --resolve --arch x86_64 nvidia-driver-cuda nvidia-driver-libs nvidia-driver-cuda-libs nvidia-persistenced --quiet > /dev/null;
rpm --install --verbose --hash --justdb /tmp/nvidia-driver-libs/* --quiet > /dev/null;
rm -rf /tmp/nvidia-driver-libs;
dnf install cuda  --assumeyes --quiet > /dev/null;
echo "export PATH=\$PATH:/usr/local/cuda/bin" >> /etc/profile.d/cuda.sh;
chmod +x /etc/profile.d/cuda.sh;
source /etc/profile.d/cuda.sh;
nvcc --version;
git clone --depth=1 https://github.com/ggerganov/llama.cpp.git /tmp/llama.cpp
cd /tmp/llama.cpp;
cmake -B build -DGGML_CUDA=ON;
cmake --build build --config Release -j 20;
cmake --install build;
cd ~;
rm -rf /tmp/llama.cpp;
echo "/usr/local/lib" | sudo tee /etc/ld.so.conf.d/local-lib.conf;
echo "/usr/local/lib64" | sudo tee /etc/ld.so.conf.d/local-lib64.conf;
ldconfig;

I think that this documentation update can be merged as it is.

[da2ce7]

I now have made a template based upon the guide provided here: https://runpod.io/console/deploy?template=mtwj86pqgc&ref=r0lfrx3d

[da2ce7]

Perhaps @ericcurtin and @ngxson would like to review this documentation update, I feel that it might have been lost in the history.

[ngxson]

Also cc @ericcurtin if you want to take a look

[ericcurtin]

Approved, Nvidia provide UBI9 based containers which are quite useful also, we use them in RamaLama