llama-cpp uses the wrong cmake flag to build for ROCM

[HiroseKoichi]

llama-cpp uses the wrong cmake flag to build for ROCM

The cmake flag to enable ROCM support in llamacpp has changed from GGML_HIPBLAS to GGML_HIP. The llama-cpp package in Nixos still uses GGML_HIPBLAS to build on ROCM enabled systems, which results in llamacpp being built with only CPU support:

warning: not compiled with GPU offload support, --gpu-layers option will be ignored
warning: see main README.md for information on enabling GPU BLAS support

Steps To Reproduce

Steps to reproduce the behavior:

1. Add llama-cpp to environment.systemPackages in configuartion.nix
2. Rebuild the configuration with nixos-rebuild switch
3. Attempt to run a model using llama-server, which notifies you that llamacpp was not built with GPU support

Expected behavior

Llama-cpp being built with GPU support on AMD hardware.

Additional context

A workaround is to add an overlay to the package to include the new cmake flag:

((pkgs.llama-cpp.overrideAttrs (finalAttrs: previousAttrs: {
cmakeFlags = (previousAttrs.cmakeFlags ++ [ "-DGGML_HIP=ON" ]);
})).override { rocmSupport = true; })

This results in llama-cpp successfully being built with GPU support on AMD hardware.

Metadata

• system: "x86_64-linux"
• host os: Linux 6.12.2-zen1, NixOS, 24.11 (Vicuna), 24.11.710905.a0f3e10d9435
• multi-user?: yes
• sandbox: yes
• version: nix-env (Nix) 2.24.10
• channels(root): "nixos-24.11"
• nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos

Notify maintainers

@dit7ya, @elohmeier, @philiptaron, @xddxdd

---

Note for maintainers: Please tag this issue in your PR.

---

Add a 👍 reaction to issues you find important.