Undefined symbol on MacOS with strict linking

[jeroen]

Compiling on MacOS without the -undefined dynamic_lookup linker flag reveals an undefined symbol. You can test this like so:

sed -i.bak 's/-undefined dynamic_lookup//g' $(R RHOME)/etc/Makeconf

And then installing from source will fail:

** configured file: 'R/tbb-autodetected.R.in' => 'R/tbb-autodetected.R'
  *** configured file: 'src/install.libs.R.in' => 'src/install.libs.R'
  *** configured file: 'src/Makevars.in' => 'src/Makevars'
  ** finished configure for package 'RcppParallel'
  ** libs
  using C++ compiler: ‘Apple clang version 14.0.0 (clang-1400.0.29.202)’
  using SDK: ‘’
  (tbb) Building TBB using bundled sources ...
  OS: macos
  arch=intel64
  compiler=clang
  runtime=cc14.0.0_os12.7.2
  tbb_build_prefix=macos_intel64_clang_cc14.0.0_os12.7.2
  work_dir=/private/var/folders/qv/pdh5wsgn0lq3dp77zj602b5c0000gn/T/RtmpXzRtgl/R.INSTALL3d205139f0cb/RcppParallel/src/build/macos_intel64_clang_cc14.0.0_os12.7.2_release
  (tbb) TBB compilation finished successfully.
  clang++ -arch x86_64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include  -I/opt/R/x86_64/include   -std=gnu++11 -DRCPP_PARALLEL_USE_TBB=1 -DTBB_SUPPRESS_DEPRECATED_MESSAGES=1 -fPIC  -falign-functions=64 -Wall -g -O2  -c init.cpp -o init.o
  clang++ -arch x86_64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include  -I/opt/R/x86_64/include   -std=gnu++11 -DRCPP_PARALLEL_USE_TBB=1 -DTBB_SUPPRESS_DEPRECATED_MESSAGES=1 -fPIC  -falign-functions=64 -Wall -g -O2  -c options.cpp -o options.o
  clang++ -arch x86_64 -std=gnu++17 -dynamiclib -Wl,-headerpad_max_install_names -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/x86_64/lib -o RcppParallel.so init.o options.o -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
  Undefined symbols for architecture x86_64:
    "tbb::interface7::internal::task_arena_base::internal_max_concurrency(tbb::interface7::task_arena const*)", referenced from:
        _defaultNumThreads in options.o
  ld: symbol(s) not found for architecture x86_64
  clang: error: linker command failed with exit code 1 (use -v to see invocation)
  make: *** [RcppParallel.so] Error 1
  ERROR: compilation failed for package ‘RcppParallel’

This is important when cross compiling (for example on p3m), otherwise the resulting binary cannot be loaded.

[eddelbuettel]

Can you put a PR together, and better yet, test it in the appropriate setting (i.e. with such cross-compilation) ?

[kevinushey]

The problem here, I suspect, is that we don't explicitly link to libtbb; instead, we just try to dynamically load it when RcppParallel is loaded. This happens here:

RcppParallel/R/zzz.R

Lines 13 to 42 in 6f81716

	loadTbbLibrary <- function(name) {
	path <- tbbLibraryPath(name)
	if (is.null(path))
	return(NULL)
	if (!file.exists(path)) {
	warning("TBB library ", shQuote(name), " not found.")
	return(NULL)
	}
	dyn.load(path, local = FALSE, now = TRUE)
	}
	.onLoad <- function(libname, pkgname) {
	# load tbb, tbbmalloc
	.tbbDllInfo <<- loadTbbLibrary("tbb")
	.tbbMallocDllInfo <<- loadTbbLibrary("tbbmalloc")
	# load tbbmalloc_proxy, but only if requested
	useTbbMallocProxy <- Sys.getenv("RCPP_PARALLEL_USE_TBBMALLOC_PROXY", unset = "FALSE")
	if (useTbbMallocProxy %in% c("TRUE", "True", "true", "1"))
	.tbbMallocProxyDllInfo <<- loadTbbLibrary("tbbmalloc_proxy")
	# load RcppParallel library if available
	.dllInfo <<- library.dynam("RcppParallel", pkgname, libname)
	}

We could probably set PKG_LIBS in some appropriate way for the "embedded" TBB case as well?

RcppParallel/src/Makevars.in

Lines 14 to 17 in 6f81716

	# If TBB_LIB is defined, link to that explicitly.
	ifdef TBB_LIB
	PKG_LIBS = -Wl,-L,"$(TBB_LIB)" -Wl,-rpath,"$(TBB_LIB)" -ltbb -ltbbmalloc
	endif

The complication here is that we don't move the library to its final install location until install.libs.R is invoked, and that happens after the linker step...

kevin@MBP-P2MQ:~/r/pkg/RcppParallel [master]
$ R CMD INSTALL --preclean .
* installing to library ‘/Users/kevin/Library/R/arm64/4.3/library’
* installing *source* package ‘RcppParallel’ ...
** using staged installation
** preparing to cleanup package 'RcppParallel' ...
*** removed file 'R/tbb-autodetected.R'
*** removed file 'src/Makevars'
*** removed file 'src/install.libs.R'
** finished cleanup for package 'RcppParallel'
** preparing to configure package 'RcppParallel' ...
*** configured file: 'R/tbb-autodetected.R.in' => 'R/tbb-autodetected.R'
*** configured file: 'src/Makevars.in' => 'src/Makevars'
*** configured file: 'src/install.libs.R.in' => 'src/install.libs.R'
** finished configure for package 'RcppParallel'
** libs
using C++ compiler: ‘Apple clang version 15.0.0 (clang-1500.0.40.1)’
using SDK: ‘MacOSX14.2.sdk’
(tbb) Building TBB using bundled sources ...
OS: macos
arch=arm64
compiler=clang
runtime=cc15.0.0_os14.2.1
tbb_build_prefix=macos_arm64_clang_cc15.0.0_os14.2.1
work_dir=/Users/kevin/r/pkg/RcppParallel/src/build/macos_arm64_clang_cc15.0.0_os14.2.1_release
(tbb) TBB compilation finished successfully.
clang++ -arch arm64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include  -I/opt/R/arm64/include   -std=gnu++11 -DRCPP_PARALLEL_USE_TBB=1 -DTBB_SUPPRESS_DEPRECATED_MESSAGES=1 -fPIC  -falign-functions=64 -Wall -g -O2  -c init.cpp -o init.o
clang++ -arch arm64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include  -I/opt/R/arm64/include   -std=gnu++11 -DRCPP_PARALLEL_USE_TBB=1 -DTBB_SUPPRESS_DEPRECATED_MESSAGES=1 -fPIC  -falign-functions=64 -Wall -g -O2  -c options.cpp -o options.o
clang++ -arch arm64 -std=gnu++17 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o RcppParallel.so init.o options.o -L/opt/homebrew/opt/libomp/lib -lomp -L/opt/homebrew/lib -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
installing via 'install.libs.R' to /Users/kevin/Library/R/arm64/4.3/library/00LOCK-RcppParallel/00new/RcppParallel
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (RcppParallel)

[eddelbuettel]

Also for what it is worth, when I am building 'water tight' packages for r2u I have to add libtbb-dev (or libtbb2-dev on 22.04) to the build-dependencies so that the shared linker-derived actual dependencies can be computed. So we do kinda sorta have a dependency on libtbb.

[jeroen]

@kevinushey setting PKG_LIBS to link to the bundled libtbb would probably solve the build for RcppParallel itself. If I look at the build log on Windows, it does seem to do this already (-Ltbb/build/lib_release -ltbb -ltbbmalloc), so perhaps this can be mimicked for macos.

...
 g++ -std=gnu++17 -shared -s -static-libgcc -o RcppParallel.dll tmp.def init.o options.o -Ltbb/build/lib_release -ltbb -ltbbmalloc -LC:/rtools43/x86_64-w64-mingw32.static.posix/lib/x64 -LC:/rtools43/x86_64-w64-mingw32.static.posix/lib -LC:/R/bin/x64 -lR

In order to also fix packages that use LinkingTo: RcppParallel and subsequently are calling libtbb functions, we need to make sure RcppParallel::LdFlags() also gives -L{path-to-RcppParallel}/lib -ltbb -ltbb_malloc when RcppParalllel was built with the bundled libtbb.

Alternatively, we could probably provide a static libtbb for MacOS via CRAN, so that you don't need the bundled one...