Fix installation location for libraries

[s9105947]

Dear Maintainer,

The install path for warpx libraries of the current development branch uses both CMAKE_INSTALL_PREFIX and CMAKE_INSTALL_LIBDIR. This causes the install step to fail if this directory can't be created.

However, as dumped by ./cmake/WarpXFunctions.cmake the variable CMAKE_INSTALL_LIBDIR already contains a full path,
and no other installation in warpx uses both prefix and install dir variables. (e.g. here)

During configuration the installation location for libraries is given by dumping the cmake variable CMAKE_INSTALL_LIBDIR.

This PR adjusts the installation of WarpX libraries (WarpX_LIB=ON) to respect this setting by removing the use of the CMAKE_INSTALL_PREFIX.

Note: This emerged when hacking a dirty package for nix, I can provide a derivation expression for reproduction if you need that.

[ax3l]

Thank you for packaging WarpX for nix! That's cool, can you please share a few more details?

Note that we currently build a couple of binary variants, mainly for 1D, 2D, 3D and RZ geometry of the code.

Please excuse the delay due to the Thanksgiving break. I'll get back to this ASAP.

[s9105947]

"Package" might be an overstatement, my attempt is much less organized than any published package you are going to encounter in the wild.

For context: Nix is a langauge ("the expression language nix") used in a package manager ("nix"), also it's the associated repository ("nixpkgs"), and an operating system ("NixOS"). (see also: How Nix works)
It is organized around the idea of reproducible builds, and the feature I'm trying to exploit is the creation of reproducible development environments.

Provided a file named shell.nix is in your current directory, invoking nix-shell reads that file, loads (and if necessary builds) all dependencies specified by it, opening a new shell -- a .profile file on steroids, if you will.
I'm tinkering with this approach for various projects, but the stuff I write is far from production-ready.

Place this is a file name shell.nix...:

{pkgs ? import <nixpkgs> {}}:
with pkgs;
let 
    warpx = stdenv.mkDerivation rec {
        pname = "warpx";
        version = "21.11";

        src = fetchurl {
            url = "https://github.com/ECP-WarpX/WarpX/archive/refs/tags/${version}.tar.gz";
            sha256 = "02qdq58r4yvn4akcgbd5ai5x3d941d8d671mfwx06cn7f5vkfq6f";
        };
        dontUseSetuptoolsBuild = true; # prevent auto-detection of setup.py as build system

        amrexSrc = fetchzip {
            url = "https://github.com/AMReX-Codes/amrex/releases/download/21.11/amrex-21.11.tar.gz";
            sha256 = "1i3qzf9azlphixggr0asfjad5c60khm05wfy9gcqyi5vp4hsld8y";
        };

        picsarSrc = fetchgit {
            url = "https://github.com/ECP-WarpX/picsar";
            rev = "8ff12fbf118b9aba7cfe602cb1a5e6da32bf7eef";
            sha256 = "0fvhq9kfgqf4zz58gd5gv7kl939w1rhf09z7mkvmk8ygsi75726n";
        };

        patches = [
            (builtins.toFile "fixLibInstall.patch" ''
                diff --git a/CMakeLists.txt b/CMakeLists.txt
                index d0dc0262..c6a7611f 100644
                --- a/CMakeLists.txt
                +++ b/CMakeLists.txt
                @@ -345,7 +345,7 @@ if(WarpX_LIB)
                     endif()
                     install(CODE "file(CREATE_LINK
                         $<TARGET_FILE_NAME:shared>
                -        ${"$"}{CMAKE_INSTALL_PREFIX}/${"$"}{CMAKE_INSTALL_LIBDIR}/libwarpx.${"$"}{lib_suffix}.${"$"}{mod_ext}
                +        ${"$"}{CMAKE_INSTALL_LIBDIR}/libwarpx.${"$"}{lib_suffix}.${"$"}{mod_ext}
                         COPY_ON_ERROR SYMBOLIC)")
                 endif()
            '')
        ];

        depsBuildBuild = [
            stdenv.cc
            cmake
        ];

        buildInputs = [
            boost
        ];

        cmakeFlags = [
            "-DWarpX_APP=ON"
            "-DWarpX_COMPUTE=OMP"
            "-DWarpX_MPI=OFF"
            "-DWarpX_LIB=ON"
            "-DWarpX_PRECISION=SINGLE"
            "-DWarpX_DIMS=3"
            "-DBUILD_SHARED_LIBS=ON"
            "-DWarpX_amrex_src=${amrexSrc}"
            "-DWarpX_picsar_src=${picsarSrc}"
        ];
    };
in
mkShell {
    buildInputs = [
        warpx
    ];
}

...and then invoke nix-shell (or the equivalent, e.g. ./path/to/nix-portable nix-shell if using nix portable) to build my custom version, with almost everything disabled (also note the already applied patch in the code above).

Then run as usual:

mkdir -p warpx_test_run
cp $(which warpx.3d.NOMPI.OMP.SP.QED) warpx.3d.NOMPI.OMP.SP.QED
wget https://warpx.readthedocs.io/en/latest/_downloads/219592f3f9af7525a069a613a5452c80/inputs_3d_boost
./warpx.3d.NOMPI.OMP.SP.QED inputs_3d_boost

[ax3l]

Quickly checked and you are right:
https://cmake.org/pipermail/cmake/2012-February/049206.html

also we don't do the same mistake on other locations of WarpX.
Will fix also in ImpactX (HiPACE++ not affected).

Thanks a lot for the report & fix 💖

[ax3l]

Uff, it looks like this caused a regression on macOS with Spack, where the post-install symlink is now not created anymore. Investigating.

https://dev.azure.com/ECP-WarpX/WarpX/_build/results?buildId=6158&view=results

• the same problem exists with Spack and Ubuntu with this patch

[ax3l]

However, as dumped by ./cmake/WarpXFunctions.cmake the variable CMAKE_INSTALL_LIBDIR already contains a full path,
and no other installation in warpx uses both prefix and install dir variables. (e.g. here)

I am not sure what happens in nix, maybe there it is set explicitly when nix calls CMake, but by default this does not give me a an absolute path printed out but instead just a relative path.

$ cmake -S . -B build
...
WarpX build configuration:
  Version: 21.12 (21.12-7-g0f0823af3f1f)
  C++ Compiler: GNU 9.3.0 
    /usr/bin/c++

  Installation prefix: /usr/local
        bin: bin
        lib: lib
    include: include
      cmake: lib/cmake/WarpX

  Build type: Release
  Build options:
...

[s9105947]

My bad, I did not read the cmake manual carefully. >_<

See my comment on #2626 for my notes.

[ax3l]

This happens, no worries :)

When you have the time, please confirm if the follow-up patch in #2626 works for your original nix problem as well :)