Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

libglvnd, ocl-icd, vulkan-loader: Add /run/opengl-driver(-32) to RUNPATH. #60985

Merged
merged 3 commits into from
May 22, 2019
Merged

libglvnd, ocl-icd, vulkan-loader: Add /run/opengl-driver(-32) to RUNPATH. #60985

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2874e84
addOpenGLRunpath: Add new hook for setting RUNPATH.
abbradar May 22, 2019
1860e50
libglvnd, ocl-icd, vulkan-loader: Add driver library paths to RUNPATH.
ambrop72 May 22, 2019
28a0918
Add driver library path to some packages to find CUDA libraries.
ambrop72 May 22, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions pkgs/applications/misc/blender/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
, ilmbase, libXi, libX11, libXext, libXrender
, libjpeg, libpng, libsamplerate, libsndfile
, libtiff, libGLU_combined, openal, opencolorio, openexr, openimageio, openjpeg_1, pythonPackages
, zlib, fftw, opensubdiv, freetype, jemalloc, ocl-icd
, zlib, fftw, opensubdiv, freetype, jemalloc, ocl-icd, addOpenGLRunpath
, jackaudioSupport ? false, libjack2
, cudaSupport ? config.cudaSupport or false, cudatoolkit
, colladaSupport ? true, opencollada
Expand All @@ -21,8 +21,9 @@ stdenv.mkDerivation rec {
sha256 = "1g4kcdqmf67srzhi3hkdnr4z1ph4h9sza1pahz38mrj998q4r52c";
};

nativeBuildInputs = [ cmake ] ++ optional cudaSupport addOpenGLRunpath;
buildInputs =
[ boost cmake ffmpeg gettext glew ilmbase
[ boost ffmpeg gettext glew ilmbase
libXi libX11 libXext libXrender
freetype libjpeg libpng libsamplerate libsndfile libtiff libGLU_combined openal
opencolorio openexr openimageio openjpeg_1 python zlib fftw jemalloc
Expand Down Expand Up @@ -80,6 +81,15 @@ stdenv.mkDerivation rec {
--prefix PYTHONPATH : ${pythonPackages.numpy}/${python.sitePackages}
'';

# Set RUNPATH so that libcuda and libnvrtc in /run/opengl-driver(-32)/lib can be
# found. See the explanation in libglvnd.
postFixup = optionalString cudaSupport ''
for program in $out/bin/blender $out/bin/.blender-wrapped; do
isELF "$program" || continue
addOpenGLRunpath "$program"
done
'';

meta = with stdenv.lib; {
description = "3D Creation/Animation/Publishing System";
homepage = https://www.blender.org;
Expand Down
12 changes: 12 additions & 0 deletions pkgs/build-support/add-opengl-runpath/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{ lib, stdenv }:

stdenv.mkDerivation {
name = "add-opengl-runpath";

driverLink = "/run/opengl-driver" + lib.optionalString stdenv.isi686 "-32";

buildCommand = ''
mkdir -p $out/nix-support
substituteAll ${./setup-hook.sh} $out/nix-support/setup-hook
'';
}
28 changes: 28 additions & 0 deletions pkgs/build-support/add-opengl-runpath/setup-hook.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Set RUNPATH so that driver libraries in /run/opengl-driver(-32)/lib can be found.
# This is needed to not rely on LD_LIBRARY_PATH which does not work with setuid
# executables. Fixes https://github.com/NixOS/nixpkgs/issues/22760. It must be run
# in postFixup because RUNPATH stripping in fixup would undo it. Note that patchelf
# actually sets RUNPATH not RPATH, which applies only to dependencies of the binary
# it set on (including for dlopen), so the RUNPATH must indeed be set on these
# libraries and would not work if set only on executables.
addOpenGLRunpath() {
Copy link
Member

@matthewbauer matthewbauer May 23, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be a no-op on macOS. We can link to the OpenGL framework there I think.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be a no-op, or we should link to the OpenGL framework?

local forceRpath=

while [ $# -gt 0 ]; do
case "$1" in
--) shift; break;;
--force-rpath) shift; forceRpath=1;;
--*)
echo "addOpenGLRunpath: ERROR: Invalid command line" \
"argument: $1" >&2
return 1;;
*) break;;
esac
done

for file in "$@"; do
local origRpath="$(patchelf --print-rpath "$file")"
patchelf --set-rpath "@driverLink@/lib:$origRpath" ${forceRpath:+--force-rpath} "$file"
done
}

14 changes: 12 additions & 2 deletions pkgs/development/compilers/cudatoolkit/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
{ lib, stdenv, makeWrapper, fetchurl, requireFile, perl, ncurses5, expat, python27, zlib
, gcc48, gcc49, gcc5, gcc6, gcc7
, xorg, gtk2, gdk_pixbuf, glib, fontconfig, freetype, unixODBC, alsaLib, glibc
, addOpenGLRunpath
}:

let
Expand Down Expand Up @@ -39,7 +40,7 @@ let

outputs = [ "out" "lib" "doc" ];

nativeBuildInputs = [ perl makeWrapper ];
nativeBuildInputs = [ perl makeWrapper addOpenGLRunpath ];
buildInputs = [ gdk_pixbuf ]; # To get $GDK_PIXBUF_MODULE_FILE via setup-hook
runtimeDependencies = [
ncurses5 expat python zlib glibc
Expand Down Expand Up @@ -143,10 +144,19 @@ let
else
rpath2=$rpath:$lib/lib:$out/jre/lib/amd64/jli:$out/lib:$out/lib64:$out/nvvm/lib:$out/nvvm/lib64
fi
patchelf --set-rpath $rpath2 --force-rpath $i
patchelf --set-rpath "$rpath2" --force-rpath $i
done < <(find $out $lib $doc -type f -print0)
'';

# Set RPATH so that libcuda and other libraries in
# /run/opengl-driver(-32)/lib can be found. See the explanation in
# addOpenGLRunpath. Don't try to figure out which libraries really need
# it, just patch all (but not the stubs libraries). Note that
# --force-rpath prevents changing RPATH (set above) to RUNPATH.
postFixup = ''
addOpenGLRunpath --force-rpath {$out,$lib}/lib/lib*.so
'';

# cuda-gdb doesn't run correctly when not using sandboxing, so
# temporarily disabling the install check. This should be set to true
# when we figure out how to get `cuda-gdb --version` to run correctly
Expand Down
18 changes: 11 additions & 7 deletions pkgs/development/libraries/libglvnd/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
{ stdenv, lib, fetchFromGitHub, fetchpatch, autoreconfHook, python2, pkgconfig, libX11, libXext, xorgproto }:
{ stdenv, lib, fetchFromGitHub, fetchpatch, autoreconfHook, python2, pkgconfig, libX11, libXext, xorgproto, addOpenGLRunpath }:

let
driverLink = "/run/opengl-driver" + lib.optionalString stdenv.isi686 "-32";
in stdenv.mkDerivation rec {
stdenv.mkDerivation rec {
name = "libglvnd-${version}";
version = "1.0.0";

Expand All @@ -13,7 +11,7 @@ in stdenv.mkDerivation rec {
sha256 = "1a126lzhd2f04zr3rvdl6814lfl0j077spi5dsf2alghgykn5iif";
};

nativeBuildInputs = [ autoreconfHook pkgconfig python2 ];
nativeBuildInputs = [ autoreconfHook pkgconfig python2 addOpenGLRunpath ];
buildInputs = [ libX11 libXext xorgproto ];

postPatch = lib.optionalString stdenv.isDarwin ''
Expand All @@ -26,7 +24,7 @@ in stdenv.mkDerivation rec {
NIX_CFLAGS_COMPILE = [
"-UDEFAULT_EGL_VENDOR_CONFIG_DIRS"
# FHS paths are added so that non-NixOS applications can find vendor files.
"-DDEFAULT_EGL_VENDOR_CONFIG_DIRS=\"${driverLink}/share/glvnd/egl_vendor.d:/etc/glvnd/egl_vendor.d:/usr/share/glvnd/egl_vendor.d\""
"-DDEFAULT_EGL_VENDOR_CONFIG_DIRS=\"${addOpenGLRunpath.driverLink}/share/glvnd/egl_vendor.d:/etc/glvnd/egl_vendor.d:/usr/share/glvnd/egl_vendor.d\""
] ++ lib.optional stdenv.cc.isClang "-Wno-error";

# Indirectly: https://bugs.freedesktop.org/show_bug.cgi?id=35268
Expand All @@ -45,7 +43,13 @@ in stdenv.mkDerivation rec {
});
outputs = [ "out" "dev" ];

passthru = { inherit driverLink; };
# Set RUNPATH so that driver libraries in /run/opengl-driver(-32)/lib can be found.
# See the explanation in addOpenGLRunpath.
postFixup = ''
addOpenGLRunpath $out/lib/libGLX.so $out/lib/libEGL.so
'';

passthru = { inherit (addOpenGLRunpath) driverLink; };

meta = with stdenv.lib; {
description = "The GL Vendor-Neutral Dispatch library";
Expand Down
12 changes: 9 additions & 3 deletions pkgs/development/libraries/ocl-icd/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{stdenv, fetchurl, ruby, opencl-headers, libGL_driver }:
{stdenv, fetchurl, ruby, opencl-headers, addOpenGLRunpath }:

stdenv.mkDerivation rec {
name = "ocl-icd-${version}";
Expand All @@ -9,12 +9,18 @@ stdenv.mkDerivation rec {
sha256 = "0f14gpa13sdm0kzqv5yycp4pschbmi6n5fj7wl4ilspzsrqcgqr2";
};

nativeBuildInputs = [ ruby ];
nativeBuildInputs = [ ruby addOpenGLRunpath ];

buildInputs = [ opencl-headers ];

postPatch = ''
sed -i 's,"/etc/OpenCL/vendors","${libGL_driver.driverLink}/etc/OpenCL/vendors",g' ocl_icd_loader.c
sed -i 's,"/etc/OpenCL/vendors","${addOpenGLRunpath.driverLink}/etc/OpenCL/vendors",g' ocl_icd_loader.c
'';

# Set RUNPATH so that driver libraries in /run/opengl-driver(-32)/lib can be found.
# See the explanation in addOpenGLRunpath.
postFixup = ''
addOpenGLRunpath $out/lib/libOpenCL.so
'';

meta = with stdenv.lib; {
Expand Down
9 changes: 9 additions & 0 deletions pkgs/development/libraries/science/math/cudnn/generic.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
, lib
, cudatoolkit
, fetchurl
, addOpenGLRunpath
}:

stdenv.mkDerivation rec {
Expand All @@ -19,6 +20,8 @@ stdenv.mkDerivation rec {
inherit sha256;
};

nativeBuildInputs = [ addOpenGLRunpath ];

installPhase = ''
function fixRunPath {
p=$(patchelf --print-rpath $1)
Expand All @@ -31,6 +34,12 @@ stdenv.mkDerivation rec {
cp -a lib64 $out/lib64
'';

# Set RUNPATH so that libcuda in /run/opengl-driver(-32)/lib can be found.
# See the explanation in addOpenGLRunpath.
postFixup = ''
addOpenGLRunpath $out/lib/lib*.so
'';

propagatedBuildInputs = [
cudatoolkit
];
Expand Down
8 changes: 6 additions & 2 deletions pkgs/development/libraries/science/math/nccl/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{ stdenv, fetchFromGitHub, which, cudatoolkit }:
{ stdenv, fetchFromGitHub, which, cudatoolkit, addOpenGLRunpath }:

stdenv.mkDerivation rec {
name = "nccl-${version}-cuda-${cudatoolkit.majorVersion}";
Expand All @@ -13,7 +13,7 @@ stdenv.mkDerivation rec {

outputs = [ "out" "dev" ];

nativeBuildInputs = [ which ];
nativeBuildInputs = [ which addOpenGLRunpath ];

buildInputs = [ cudatoolkit ];

Expand All @@ -28,6 +28,10 @@ stdenv.mkDerivation rec {

postFixup = ''
moveToOutput lib/libnccl_static.a $dev
# Set RUNPATH so that libnvidia-ml in /run/opengl-driver(-32)/lib can be found.
# See the explanation in addOpenGLRunpath.
addOpenGLRunpath $out/lib/lib*.so
'';

NIX_CFLAGS_COMPILE = [ "-Wno-unused-function" ];
Expand Down
12 changes: 9 additions & 3 deletions pkgs/development/libraries/vulkan-loader/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{ stdenv, fetchFromGitHub, cmake, python3, vulkan-headers, pkgconfig
, xlibsWrapper, libxcb, libXrandr, libXext, wayland, libGL_driver }:
, xlibsWrapper, libxcb, libXrandr, libXext, wayland, addOpenGLRunpath }:

let
version = "1.1.106";
Expand All @@ -17,17 +17,23 @@ stdenv.mkDerivation rec {
sha256 = "0zhrwj1gi90x2w8gaaaw5h4b969a8gfy244kn0drrplhhb1nqz3b";
};

nativeBuildInputs = [ pkgconfig ];
nativeBuildInputs = [ pkgconfig addOpenGLRunpath ];
buildInputs = [ cmake python3 xlibsWrapper libxcb libXrandr libXext wayland ];
enableParallelBuilding = true;

cmakeFlags = [
"-DFALLBACK_DATA_DIRS=${libGL_driver.driverLink}/share:/usr/local/share:/usr/share"
"-DFALLBACK_DATA_DIRS=${addOpenGLRunpath.driverLink}/share:/usr/local/share:/usr/share"
"-DVULKAN_HEADERS_INSTALL_DIR=${vulkan-headers}"
];

outputs = [ "out" "dev" ];

# Set RUNPATH so that driver libraries in /run/opengl-driver(-32)/lib can be found.
# See the explanation in addOpenGLRunpath.
postFixup = ''
addOpenGLRunpath $out/lib/libvulkan.so
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should not be needed.

Vulkan Loader doesn't need the openGLRunpath hardcoded because it will look up libraries not in the opengl path but in whatever the ICD json points to. So vulkan will never look in /run/opengl-driver/lib,
it will instead look in /nix/store/y99l7729c0157a52y9qfzi3zg4f94b57-mesa-noglu-18.3.4-drivers/lib/libvulkan_intel.so because the ICD in /run/opengl-driver/share/vulkan/icd.d says so

cat /run/opengl-driver/share/vulkan/icd.d/intel_icd.x86_64.json 
{
    "ICD": {
        "api_version": "1.1.90",
        "library_path": "/nix/store/y99l7729c0157a52y9qfzi3zg4f94b57-mesa-noglu-18.3.4-drivers/lib/libvulkan_intel.so"
    },
    "file_format_version": "1.0.0"
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is currently needed for NVidia because the library_path is only a name:

$ cat /run/opengl-driver-32/share/vulkan/icd.d/nvidia.json 
{
    "file_format_version" : "1.0.0",
    "ICD": {
        "library_path": "libGLX_nvidia.so",
        "api_version" : "1.1.99"
    }
}

I think we should patch it and then remove the runpath.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. any other drivers that would need similar fixups? ;-)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opened #62870 to set absolute paths in NVidia ICD files.
Also related:

'';

meta = with stdenv.lib; {
description = "LunarG Vulkan loader";
homepage = https://www.lunarg.com;
Expand Down
2 changes: 2 additions & 0 deletions pkgs/top-level/all-packages.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,8 @@ in
}
'');

addOpenGLRunpath = callPackage ../build-support/add-opengl-runpath { };

# Zip file format only allows times after year 1980, which makes e.g. Python wheel building fail with:
# ValueError: ZIP does not support timestamps before 1980
ensureNewerSourcesForZipFilesHook = ensureNewerSourcesHook { year = "1980"; };
Expand Down