Tracking issue: mesa on darwin

[yu-re-ka]

Currently the situation of the mesa package on darwin is not optimal. This is because we are lugging around Mesa 22.3 just for darwin. We would like to remove Mesa 22.3 and find another solution in time for the NixOS 23.11 release cycle.

Several options were discussed in the Nixpkgs Dev matrix room:

• Get current mesa (23.1) to build for darwin; currently blocked by upstream issue ; upstream contributions are most likely welcome
• Gate all places that depend on mesa with !isDarwin or availableOn if they can build without it
  Some common dependency chains seem to be gtk3-x11 -> libepoxy -> libgl and graphviz -> libdevil -> libgl.

Anything that uses mesa on darwin is being rendered purely using the CPU using llvmpipe anyways, since mesa does not have any driver backends that work on darwin.

Probably a combination of both reducing unreasonable dependencies on mesa, and making the latest version build, would be good

CC @NixOS/darwin-maintainers

[yu-re-ka]

In #233220 we have removed the 22.3 version, marked mesa as broken on darwin, and removed some key dependents. Help is welcome for upstream fixes to unbreak mesa (upstream issue), and removing further unnecessary dependents.

[reckenrode]

I saw this come up in the Mesa discussion. I’d like to look at it (though I don’t think Mesa should be built on Darwin, so I’d be looking at how to remove it as a dependency where possible), but I’m currently focused on the LLVM update for the Darwin stdenv (now tracked at #234710).

[reckenrode]

This is a hacky patch I put together that provides OpenGL.framework and a pkg-config wrapper for OpenGL.framework. I was able to build libGLU with it as well as a few Linux-only derivations. Anything that expects GLX (like FreeGLUT) won’t work. Not sure if this is the solution, but it does at least take mesa out of the picture.

[iFreilicht]

I would like to go the opposite direction and enable mesa on darwin again. There are quite a few packages that depend on this, I'm specifically looking at python packages that do rendering with libGL and rely on pyglet, moderngl or screeninfo for that. Granted those are marked broken on darwin right now as well, but I would like to fix that, and if mesa is removed from nixpkgs, that would bar me from doing that work.

I'm uncertain if it's easily possible to build libGL for darwin separately, but that might be useful in its own right.

@yu-re-ka you meantioned in the other ticket:

I even created a hack that would have made mesa 23.1 build on darwin

do you still have the code for that?

[reckenrode]

Those packages don’t require Mesa. They support Darwin upstream and should be able to link against the system OpenGL.framework. I was able to build moderngl and pyglet but not screeninfo using the patch I linked in my previous reply. See below for details.

Note: Darwin was added to meta.platforms and removed from meta.broken as necessary.

• moderngl: Make libGLX conditional on platform. Add the ApplicationsServices framework to the buildInputs of glcontext (a dependency) and make libGLX conditional.
• pyglet: Modify load_library to be appropriate for Darwin.
• screeninfo: Requires Cython, but it won’t build once that’s added to buildInputs on Darwin because it also requires pyobjc-framework-Cocoa (see Enabling pyobjc on python3Packages #101360 and Python on Darwin: pyobj tracking issue #105156).

[iFreilicht]

I see, thank you for the explanation. I'd be willing to try and turn this into a less hacky solution that is fit to be merged.
My approach would be to split libGL into two platform-specific packages and one meta-package that automatically chooses the correct version depending on host-platform, and mention that in the description of the meta-package. I think that would prevent confusion about different versions when searching on search.nixos.org.

If you'd rather make these changes yourself (which honestly might be quicker), I'd be happy to review and test them.

[emilazy]

The libGL package is theoretically already meant to basically be this, but even on Darwin it relies on Mesa stub stuff that is broken right now.

I think it would be great to have packages generally depend on something that supplies the native GL implementation rather than mesa directly. Though I'd also like it if mesa built on Darwin just on general principles, even if it usually shouldn't be used.

[reckenrode]

I agree with the above, though I’d like to see libGL provide OpenGL.framework instead of using Mesa at all on Darwin. I doubt anything that supports Darwin is going to try to use OpenGL ES there since it’s never been supported.

[reckenrode]

If you'd rather make these changes yourself (which honestly might be quicker), I'd be happy to review and test them.

I’m already pretty focused on the Darwin stdenv stuff, so I’d like to defer to the maintainers for any fixes to the libGL situation. The patch I proposed is one solution, but I don’t know if it would be the preferred one.

[wn]

@reckenrode how can I apply your patch to my nix home-manager? I'm not too familiar with the internal workings on home-manager, but it seems i cant start nix home manager due to this issue
https://github.com/wn/nixfiles

[reckenrode]

@reckenrode how can I apply your patch to my nix home-manager? I'm not too familiar with the internal workings on home-manager, but it seems i cant start nix home manager due to this issue
https://github.com/wn/nixfiles

I replied in the reddit thread, but I’ll also reply here. I applied the patch and copied the resulting derivation into an overlay.

https://gist.github.com/reckenrode/dc4fb6bbcec0384a9dd3917e5af24e7f

[quentinmit]

FYI, Mesa now builds again on Darwin as of https://gitlab.freedesktop.org/mesa/mesa/-/commit/4ef573735efc7f15d8b8700622a5865d33c34bf1

AFAICT this hasn't hit a release branch yet, but with the following overlay on release-23.11 I'm able to build mesa on Darwin again:

  mesa23_3_0_main = let
    inherit (final) fetchFromGitLab fetchurl lib;
    version = "23.3.0-main";
    hash = "sha256-kHrUNnUedCAc6uOWCHdd/2LMMcz3BAqJPcXnCbHLlaw=";
    branch = lib.versions.major version;
  in prev.mesa.overrideAttrs (old: {
    inherit version;
    src = fetchFromGitLab {
      domain = "gitlab.freedesktop.org";
      owner = "mesa";
      repo = "mesa";
      rev = "4ef573735efc7f15d8b8700622a5865d33c34bf1";
      inherit hash;
    };
    # Note that the following changes should all be conditional on Darwin; for simplicity I didn't do that in this overlay.
    patches = builtins.filter (
      p: let b = builtins.baseNameOf p; in
         b != "opencl.patch"
         && b != "disk_cache-include-dri-driver-path-in-cache-key.patch"
    ) old.patches ++ [
    ];
    mesonFlags =
      (builtins.filter (f: !(lib.hasPrefix "-Ddisk-cache-key=" f)) old.mesonFlags) ++ [
        "-Dgbm=disabled"
        "-Dxlib-lease=disabled"
        "-Degl=disabled"
        "-Dgallium-vdpau=disabled"
        "-Dgallium-va=disabled"
        "-Dgallium-xa=disabled"
        "-Dlmsensors=disabled"
      ];
    meta = old.meta // {
      broken = false;
      platforms = lib.platforms.darwin;
    };
  });

[K900]

I think we can close this now, because #319047 makes Mesa/Darwin a separate code path, and #290421 should get rid of most of its usages.

[emilazy]

After #321800 and #321942 Mesa has less than 20 reverse dependencies on Darwin and doesn’t block the channel, I think we can safely say we’ve put this one to bed. Our long national nightmare is over!