libcrypt.so.1: support newer hash types via libxcrypt

[dottedmag]

This is more "intent to implement" rather than a bug report.

Describe the bug

glibc's libcrypt implementation supports a narrow range of hashes, and none of modern RAM-heavy ones.

glibc maintaines would like to spin this library off into a separate project and drop libcrypt from the glibc altogether.

Some distributions have already switched to API- and ABI-compatible libxcrypt:

• Fedora
• Arch

Expected behaviour

All packages that use libcrypt.so.1 can use strong hash types.

Action plan

• Package libxcrypt.
• Add a flag withLibcrypt (defaults to true) to glibc package that disables building and installing libcrypt.so.1 and crypt.h if set to false.
• One-by-one update packages that expect libcrypt.so.1 in glibc by giving them a flag useGlibcLibcrypt (defaults to true).
• Add a global setting useGlibcLibcrypt (defaults to true), and disable withLibgcrypt in glibc and useGlibcLibgcrypt in other packages it is set to false.
• Wait until the dust settles.
• Switch useGibcLibcrypt to default to false.
• Wait until the dust settles.
• Remove withLibcrypt, useGlibcLibcrypt from packages, make useGlibcLibcrypt no-op.

Known issues

I have stumbled upon the following problems while doing a PoC on the packages I regularly use (not the whole archive):

• I have no idea how to make the global flag be respected
• ppp expects and checks libcrypt in glibc

Affected packages

Normally it would be easy to do a reverse-dependency search to see what relies on a particular library. However with a library bundled in glibc the tree of reverse dependencies is not particularly useful.

I have greped the whole nixos-unstable and manually processed the results. Attached is the list of

• Packages with files that link to libcrypt.so.1 and actually use symbols from them (+++)
• Packages with files that link to libcrypt.so.1, but don't actually use any symbols (???)
• Packages with files that mention libcrypt.so.1, but don't link to it (mostly text) (XXX).

This is a very rough list, the only way to get a definite list is by rebuilding the archive, however it should suffice as a starting point.

libcrypt-users.txt

Notify maintainers
@edolstra

[Izorkin]

It is possible to add tcb support? - #109457

[dottedmag]

libxcrypt is already packaged in nixpkgs, so I don't see why it wouldn't be possible. See the linked linux-pam package change as a guide.

[Izorkin]

I failed to configure linux-pam and patch passwd.

[lukegb]

@dottedmag Are you still working on this?

I propose that we get a new Hydra jobset so we can smoke out these deps (i.e. we push a jobset with glibc's libcrypt disabled and see what breaks)

[dottedmag]

I took a hiatus from working on NixOS at the moment, but still intend to do it some time in the future.

[lukegb]

No worries - do you mind if I pick it up for the moment?

[dottedmag]

Sure, feel free to.

[mweinelt]

ppp expects and checks libcrypt in glibc

ppp does not depend on libcrypt from glibc anymore since 2.4.8 (ppp-project/ppp@3c7b862), so that should not be a blocker anymore.

[bbjubjub2494]

openssh appears to currently be incompatible with pamWithLibxcrypt. Unclear to me exactly why.

source

Nov 27 08:14:45 nixos systemd[860]: PAM unable to dlopen(/nix/store/jjdm85j98ih9d28hkcs8px5k218mcrpr-linux-pam-1.5.1/lib/security/pam_unix.so): /nix/store/z56jcx3j1gfyk4sv7g8iaan0ssbdkhz1-glibc-2.33-56/lib/libcrypt.so.1: version `XCRYPT_2.0' not found (required by /nix/store/jjdm85j98ih9d28hkcs8px5k218mcrpr-linux-pam-1.5.1/lib/security/pam_unix.so)
Nov 27 08:14:45 nixos systemd[860]: PAM adding faulty module: /nix/store/jjdm85j98ih9d28hkcs8px5k218mcrpr-linux-pam-1.5.1/lib/security/pam_unix.so
Nov 27 08:14:45 nixos systemd[860]: PAM failed: Module is unknown
Nov 27 08:14:45 nixos systemd[860]: user@0.service: Failed to set up PAM session: Operation not permitted
Nov 27 08:14:45 nixos systemd[860]: user@0.service: Failed at step PAM spawning /nix/store/az8yvxswppdwn9zpm377rzfjk5gwak3b-systemd-249.5/lib/systemd/systemd: Operation not permitted

[mweinelt]

@lukegb @lourkeur Anyone still working on this? Does everyone agree with the steps mentioned in the action plan?

Add a flag withLibcrypt (defaults to true) to glibc package that disables building and installing libcrypt.so.1 and crypt.h if set to false.

This is probably just --disable-crypt in configureFlags within glibc's common.nix.

[bbjubjub2494]

@mweinelt I use a quick and dirty overlay on the daily but I am not working on a proper solution. I think broadly the action plan is correct.

[mweinelt]

This is probably just --disable-crypt in configureFlags within glibc's common.nix.

diff --git a/pkgs/development/libraries/glibc/common.nix b/pkgs/development/libraries/glibc/common.nix
index 2fc89a968f8..2ea58072014 100644
--- a/pkgs/development/libraries/glibc/common.nix
+++ b/pkgs/development/libraries/glibc/common.nix
@@ -36,6 +36,7 @@
 , withLinuxHeaders ? false
 , profilingLibraries ? false
 , withGd ? false
+, withLibcrypt ? true
 , meta
 , extraBuildInputs ? []
 , extraNativeBuildInputs ? []
@@ -201,7 +202,9 @@ stdenv.mkDerivation ({
       # To avoid linking with -lgcc_s (dynamic link)
       # so the glibc does not depend on its compiler store path
       "libc_cv_as_needed=no"
-    ] ++ lib.optional withGd "--with-gd";
+    ]
+    ++ lib.optional withGd "--with-gd"
+    ++ lib.optional (!withLibcrypt) "--disable-crypt";
 
   makeFlags = [
     "OBJCOPY=${stdenv.cc.targetPrefix}objcopy"
diff --git a/pkgs/development/libraries/glibc/default.nix b/pkgs/development/libraries/glibc/default.nix
index ba782321559..d34bbdccf25 100644
--- a/pkgs/development/libraries/glibc/default.nix
+++ b/pkgs/development/libraries/glibc/default.nix
@@ -2,6 +2,7 @@
 , withLinuxHeaders ? true
 , profilingLibraries ? false
 , withGd ? false
+, withLibcrypt? false
 , buildPackages
 }:
 
@@ -16,7 +17,7 @@ in
 callPackage ./common.nix { inherit stdenv; } {
     pname = "glibc" + lib.optionalString withGd "-gd";
 
-    inherit withLinuxHeaders profilingLibraries withGd;
+    inherit withLinuxHeaders profilingLibraries withGd withLibcrypt;
 
     # Note:
     # Things you write here override, and do not add to,

[mweinelt]

We are now in staging and everything is scheduled for this going live in 22.11. https://nixpk.gs/pr-tracker.html?pr=181764

There are some additional changes (docs, release notes, deprecations) that I'm currently working over in #195271. Reviews welcome.

[mweinelt]

Fixed in #195271.