folly: 2019.11.11.00 -> 2020.09.28.00, fix build

[Sohalt]

Motivation for this change

ZHF: #97479

Things done

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Ensured that relevant documentation is up to date
• Fits CONTRIBUTING.md.

[r-burns]

Fyi, this fails for me on macos:

builder for '/nix/store/2zd1ac73w6hyl2vqcc0fzjxwfilg6fsr-folly-2020.09.28.00.drv' failed with exit code 2; last 10 log lines:
    "_xallocx", referenced from:
        folly::threadlocal_detail::StaticMetaBase::reallocate(folly::threadlocal_detail::ThreadEntry*, unsigned int, unsigned long&) in libfolly.a(ThreadLocalDetail.cpp.o)
        folly::IOBuf::reserveSlow(unsigned long, unsigned long) in libfolly.a(IOBuf.cpp.o)
        folly::fbvector<iovec, std::__1::allocator<iovec> >::reserve(unsigned long) in libfolly.a(IOBuf.cpp.o)
        void folly::fbvector<iovec, std::__1::allocator<iovec> >::emplace_back_aux<iovec>(iovec&&) in libfolly.a(IOBuf.cpp.o)
  ld: symbol(s) not found for architecture x86_64
  clang-7: error: linker command failed with exit code 1 (use -v to see invocation)

Looks like it requires the 10.13 SDK now. Never mind, looks like marsam has the correct solution

[marsam]

would you mind adding the following to fix the darwin build?

cmakeFlags = stdenv.lib.optionals stdenv.cc.isClang [ "-DCOMPILER_HAS_F_ALIGNED_NEW=OFF" ];

[r-burns]

Hmm, now I'm getting this:

  /private/var/folders/78/nbjn_b991sd60rhqbs_0xwd8x2nf6n/T/nix-build-folly-2020.09.28.00.drv-0/source/folly/concurrency/UnboundedQueue.h:561:21: note: if you supply your own aligned allocation functions, use -faligned-allocation to silence this diagnostic
  /private/var/folders/78/nbjn_b991sd60rhqbs_0xwd8x2nf6n/T/nix-build-folly-2020.09.28.00.drv-0/source/folly/concurrency/UnboundedQueue.h:565:7: error: aligned deallocation function of type 'void (void *, enum std::align_val_t) noexcept' is only available on macOS 10.13 or newer
        delete next;
        ^
  /private/var/folders/78/nbjn_b991sd60rhqbs_0xwd8x2nf6n/T/nix-build-folly-2020.09.28.00.drv-0/source/folly/concurrency/UnboundedQueue.h:565:7: note: if you supply your own aligned allocation functions, use -faligned-allocation to silence this diagnostic
  12 warnings and 18 errors generated.

[Sohalt]

I cannot test on OSX, so I cannot help here. If someone finds a fix I'm happy to push that.

[r-burns]

If this does require the 10.13 SDK, then this will break on macos until the sdk used by nixpkgs gets bumped, which could potentially be a while. So I would think that we'd either need to block this update until that happens, or keep the old 2019.11.11.00 around for use on macs.
However I'm very new to nixpkgs so don't take this as the gospel truth. I've noticed a couple other packages failing on hydra due to requiring the 10.13 sdk so maybe this isn't a blocker.

[acairncross]

Was the 2019.11.11.00 build working on Mac (it wasn't for Linux)?

[vcunat]

On current nixpkgs master the old version builds for darwin.

[risicle]

Works non-nixos linux x86_64,

fails as above on macos 10.14

[acairncross]

Is there a reason for this to be a static library, or has it just not been changed from its default? Adding "-DBUILD_SHARED_LIBS=ON" to cmakeFlags ought to build a shared library.

[Sohalt]

Is there a reason for this to be a static library, or has it just not been changed from its default? Adding "-DBUILD_SHARED_LIBS=ON" to cmakeFlags ought to build a shared library.

None that I know of. Let's ask the maintainers @abbradar @pierreis (I guess I should've pinged them way earlier…)

[domenkozar]

bump :)

[acairncross]

Considering #59846 exists, my guess is that the build has always been failing for macOS, and is non-trivial to fix, so I don't think that should be a blocker for this being merged (should meta.platforms be made to exclude Darwin?).

Edit: Actually, as mentioned at the bottom of that PR, #62330 apparently fixed the macOS build. So it's not clear to me whether 2019.11.11.00 builds or not, would be useful if someone could find out.

I am pretty sure that shared libs should be built though, so I think that ought to be changed.

[r-burns]

Sorry didn't see your question earlier, can confirm 2019.11.11.00 is currently building on macos. Agree w.r.t. shared libs.

[Sohalt]

I've added "-DBUILD_SHARED_LIBS=ON", just squash on merge.

[r-burns]

Actually, I think this builds fine on macos without jemalloc support. Maybe we can just add jemalloc to deps for linux only. Adding jemalloc to deps for 2019.11.11.00 causes similar failures, and I'm seeing some upstream issues related to this.

[veprbl]

Didn't work for me without jemalloc, but I was able to build it after a following change:

diff --git a/pkgs/development/libraries/folly/default.nix b/pkgs/development/libraries/folly/default.nix
index 128ba7e0548..c4349a75977 100644
--- a/pkgs/development/libraries/folly/default.nix
+++ b/pkgs/development/libraries/folly/default.nix
@@ -17,7 +17,7 @@
 , fmt
 }:
 
-stdenv.mkDerivation rec {
+stdenv.mkDerivation (rec {
   pname = "folly";
   version = "2020.09.28.00";
 
@@ -63,4 +63,7 @@ stdenv.mkDerivation rec {
     platforms = [ "x86_64-linux" "x86_64-darwin" ];
     maintainers = with maintainers; [ abbradar pierreis ];
   };
-}
+} // stdenv.lib.optionalAttrs stdenv.isDarwin {
+  CXXFLAGS = "-faligned-allocation";
+  LDFLAGS = "-ljemalloc";
+})

[SuperSandro2000]

@Sohalt can you incorporate the darwin patch?

[SuperSandro2000]

Result of nixpkgs-review pr 99133 run on x86_64-linux 1

1 package built:

• folly

[Sohalt]

@Sohalt can you incorporate the darwin patch?

I just did, but I cannot test it.

[r-burns]

Commits should be squashed I think but looks good otherwise

Result of nixpkgs-review pr 99133 run on x86_64-darwin 1

1 package built:

• folly

[Sohalt]

@r-burns I've squashed everything.

[veprbl]

Result of nixpkgs-review pr 99133 on x86_64-darwin 1

1 package built:

• folly

[SuperSandro2000]

Result of nixpkgs-review pr 99133 run on x86_64-darwin 1

1 package built:

• folly

[SuperSandro2000]

Result of nixpkgs-review pr 99133 run on x86_64-darwin 1

1 package built:

• folly