Skip to content

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MacOS homebrew installer - request for update with support for pybullet #1107

Closed
ahundt opened this issue May 9, 2017 · 11 comments
Closed

MacOS homebrew installer - request for update with support for pybullet #1107

ahundt opened this issue May 9, 2017 · 11 comments
Labels
new feature

Comments

@ahundt
Copy link

ahundt commented May 9, 2017 •
edited
Loading

I'm trying to update the Homebrew/homebrew-core repository with this new bullet.rb install script, which is modified to support pybullet with these changes in a ahundt/homebrew-core bullet branch, but there appears to be bugs in bullet's own CMake script for setting Python and NumPy variables on OSX, triggering CMake build error and selecting the incorrect python library version.

I attempt to install with the following command:

brew uninstall bullet --ignore-dependencies; brew install bullet --with-demo --with-double-precision --with-framework -v

The formula (aka ruby install script) repository should be in the /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula folder if homebrew is installed.

My updated version of the script generates this cmake command:

cmake
..
-DCMAKE_C_FLAGS_RELEASE=-DNDEBUG
-DCMAKE_CXX_FLAGS_RELEASE=-DNDEBUG
-DCMAKE_INSTALL_PREFIX=/usr/local/Cellar/bullet/2.86.1_1
-DCMAKE_BUILD_TYPE=Release
-DCMAKE_FIND_FRAMEWORK=LAST
-DCMAKE_VERBOSE_MAKEFILE=ON
-Wno-dev
-DHAVE_CLOCK_GETTIME:INTERNAL=0
-DINSTALL_EXTRA_LIBS=ON
-DBUILD_UNIT_TESTS=OFF
-DUSE_DOUBLE_PRECISION=ON
-DBUILD_BULLET2_DEMOS=ON
-DBUILD_PYBULLET_MAC_USE_PYTHON_FRAMEWORK=ON
-DBUILD_PYBULLET=ON
-DBUILD_PYBULLET_NUMPY=ON
-DBUILD_PYBULLET_CLSOCKET=ON
-DBUILD_PYBULLET_ENET=ON
-DPYTHON_EXECUTABLE=/usr/local/opt/python/bin/python
-DPYTHON_LIBRARIES=/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib
-DPYTHON_INCLUDE_DIR=/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/include/python2.7
-DPYTHON_NUMPY_INCLUDE_DIR=/usr/local/lib/python2.7/site-packages/numpy/core/include
-DBUILD_ENET=ON
-DBUILD_CLSOCKET=ON
-DBUILD_SHARED_LIBS=ON
-DFRAMEWORK=ON
-DCMAKE_INSTALL_PREFIX=/usr/local/Cellar/bullet/2.86.1_1/Frameworks
-DCMAKE_INSTALL_NAME_DIR=/usr/local/Cellar/bullet/2.86.1_1/Frameworks

The command above produces the linker error below. This is most likely due to the attempt at linking python 3 with /usr/local/Frameworks/Python.framework/Versions/3.6/lib/libpython3.6m.dylib despite python 2 being specified:

[ 80%] Linking CXX shared library pybullet.dylib
cd /Users/athundt/source/bullet3/build_cmake/examples/pybullet && /usr/local/Cellar/cmake/3.8.1/bin/cmake -E cmake_link_script CMakeFiles/pybullet.dir/link.txt --verbose=1
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -DNDEBUG -dynamiclib -Wl,-headerpad_max_install_names -compatibility_version 2.87.0 -current_version 2.87.0 -o pybullet.2.87.dylib -install_name /Users/athundt/source/bullet3/build_cmake/examples/pybullet/pybullet.2.87.dylib CMakeFiles/pybullet.dir/pybullet.c.o CMakeFiles/pybullet.dir/__/SharedMemory/IKTrajectoryHelper.cpp.o CMakeFiles/pybullet.dir/__/ExampleBrowser/InProcessExampleBrowser.cpp.o CMakeFiles/pybullet.dir/__/SharedMemory/TinyRendererVisualShapeConverter.cpp.o CMakeFiles/pybullet.dir/__/OpenGLWindow/SimpleCamera.cpp.o CMakeFiles/pybullet.dir/__/TinyRenderer/geometry.cpp.o CMakeFiles/pybullet.dir/__/TinyRenderer/model.cpp.o CMakeFiles/pybullet.dir/__/TinyRenderer/tgaimage.cpp.o CMakeFiles/pybullet.dir/__/TinyRenderer/our_gl.cpp.o CMakeFiles/pybullet.dir/__/TinyRenderer/TinyRenderer.cpp.o CMakeFiles/pybullet.dir/__/SharedMemory/InProcessMemory.cpp.o CMakeFiles/pybullet.dir/__/SharedMemory/PhysicsClient.cpp.o CMakeFiles/pybullet.dir/__/SharedMemory/PhysicsServer.cpp.o CMakeFiles/pybullet.dir/__/SharedMemory/PhysicsServerExample.cpp.o CMakeFiles/pybullet.dir/__/SharedMemory/SharedMemoryInProcessPhysicsC_API.cpp.o CMakeFiles/pybullet.dir/__/SharedMemory/PhysicsServerSharedMemory.cpp.o CMakeFiles/pybullet.dir/__/SharedMemory/PhysicsDirect.cpp.o CMakeFiles/pybullet.dir/__/SharedMemory/PhysicsDirectC_API.cpp.o CMakeFiles/pybullet.dir/__/SharedMemory/PhysicsServerCommandProcessor.cpp.o CMakeFiles/pybullet.dir/__/SharedMemory/PhysicsClientSharedMemory.cpp.o CMakeFiles/pybullet.dir/__/SharedMemory/PhysicsClientSharedMemory_C_API.cpp.o CMakeFiles/pybullet.dir/__/SharedMemory/PhysicsClientC_API.cpp.o CMakeFiles/pybullet.dir/__/SharedMemory/Win32SharedMemory.cpp.o CMakeFiles/pybullet.dir/__/SharedMemory/PosixSharedMemory.cpp.o CMakeFiles/pybullet.dir/__/Utils/b3ResourcePath.cpp.o CMakeFiles/pybullet.dir/__/Utils/RobotLoggingUtil.cpp.o CMakeFiles/pybullet.dir/__/ThirdPartyLibs/tinyxml/tinystr.cpp.o CMakeFiles/pybullet.dir/__/ThirdPartyLibs/tinyxml/tinyxml.cpp.o CMakeFiles/pybullet.dir/__/ThirdPartyLibs/tinyxml/tinyxmlerror.cpp.o CMakeFiles/pybullet.dir/__/ThirdPartyLibs/tinyxml/tinyxmlparser.cpp.o CMakeFiles/pybullet.dir/__/ThirdPartyLibs/Wavefront/tiny_obj_loader.cpp.o CMakeFiles/pybullet.dir/__/ThirdPartyLibs/stb_image/stb_image.cpp.o CMakeFiles/pybullet.dir/__/Importers/ImportColladaDemo/LoadMeshFromCollada.cpp.o CMakeFiles/pybullet.dir/__/Importers/ImportObjDemo/LoadMeshFromObj.cpp.o CMakeFiles/pybullet.dir/__/Importers/ImportObjDemo/Wavefront2GLInstanceGraphicsShape.cpp.o CMakeFiles/pybullet.dir/__/Importers/ImportMJCFDemo/BulletMJCFImporter.cpp.o CMakeFiles/pybullet.dir/__/Importers/ImportURDFDemo/BulletUrdfImporter.cpp.o CMakeFiles/pybullet.dir/__/Importers/ImportURDFDemo/MyMultiBodyCreator.cpp.o CMakeFiles/pybullet.dir/__/Importers/ImportURDFDemo/URDF2Bullet.cpp.o CMakeFiles/pybullet.dir/__/Importers/ImportURDFDemo/UrdfParser.cpp.o CMakeFiles/pybullet.dir/__/Importers/ImportURDFDemo/urdfStringSplit.cpp.o CMakeFiles/pybullet.dir/__/Importers/ImportMeshUtility/b3ImportMeshUtility.cpp.o CMakeFiles/pybullet.dir/__/MultiThreading/b3PosixThreadSupport.cpp.o CMakeFiles/pybullet.dir/__/MultiThreading/b3Win32ThreadSupport.cpp.o CMakeFiles/pybullet.dir/__/MultiThreading/b3ThreadSupportInterface.cpp.o CMakeFiles/pybullet.dir/__/SharedMemory/PhysicsClientUDP.cpp.o CMakeFiles/pybullet.dir/__/SharedMemory/PhysicsClientUDP_C_API.cpp.o CMakeFiles/pybullet.dir/__/ThirdPartyLibs/enet/win32.c.o CMakeFiles/pybullet.dir/__/ThirdPartyLibs/enet/unix.c.o CMakeFiles/pybullet.dir/__/ThirdPartyLibs/enet/callbacks.c.o CMakeFiles/pybullet.dir/__/ThirdPartyLibs/enet/compress.c.o CMakeFiles/pybullet.dir/__/ThirdPartyLibs/enet/host.c.o CMakeFiles/pybullet.dir/__/ThirdPartyLibs/enet/list.c.o CMakeFiles/pybullet.dir/__/ThirdPartyLibs/enet/packet.c.o CMakeFiles/pybullet.dir/__/ThirdPartyLibs/enet/peer.c.o CMakeFiles/pybullet.dir/__/ThirdPartyLibs/enet/protocol.c.o CMakeFiles/pybullet.dir/__/SharedMemory/PhysicsClientTCP.cpp.o CMakeFiles/pybullet.dir/__/SharedMemory/PhysicsClientTCP_C_API.cpp.o CMakeFiles/pybullet.dir/__/ThirdPartyLibs/clsocket/src/SimpleSocket.cpp.o CMakeFiles/pybullet.dir/__/ThirdPartyLibs/clsocket/src/ActiveSocket.cpp.o CMakeFiles/pybullet.dir/__/ThirdPartyLibs/clsocket/src/PassiveSocket.cpp.o ../ExampleBrowser/libBulletExampleBrowserLib.2.87.dylib ../../Extras/Serialize/BulletWorldImporter/libBulletWorldImporter.2.87.dylib ../../src/BulletSoftBody/libBulletSoftBody.2.87.dylib ../../Extras/InverseDynamics/libBulletInverseDynamicsUtils.2.87.dylib ../../src/BulletInverseDynamics/libBulletInverseDynamics.2.87.dylib ../OpenGLWindow/libOpenGLWindow.dylib ../ThirdPartyLibs/Gwen/libgwen.dylib ../ThirdPartyLibs/BussIK/libBussIK.dylib ../../src/Bullet3Common/libBullet3Common.2.87.dylib /usr/local/Frameworks/Python.framework/Versions/3.6/lib/libpython3.6m.dylib -framework Cocoa -framework OpenGL ../../Extras/Serialize/BulletFileLoader/libBulletFileLoader.2.87.dylib ../../src/BulletDynamics/libBulletDynamics.2.87.dylib ../../src/BulletCollision/libBulletCollision.2.87.dylib ../../src/LinearMath/libLinearMath.2.87.dylib
Undefined symbols for architecture x86_64:
  "_PyCObject_AsVoidPtr", referenced from:
      __import_array in pybullet.c.o
  "_PyCObject_Type", referenced from:
      __import_array in pybullet.c.o
  "_PyInt_FromLong", referenced from:
      _pybullet_connectPhysicsServer in pybullet.c.o
      _pybullet_loadSDF in pybullet.c.o
      _pybullet_loadBullet in pybullet.c.o
      _pybullet_loadMJCF in pybullet.c.o
      _pybullet_getNumBodies in pybullet.c.o
      _pybullet_getBodyUniqueId in pybullet.c.o
      _pybullet_getNumConstraints in pybullet.c.o
      ...
  "_PyString_FromString", referenced from:
      _pybullet_getBodyInfo in pybullet.c.o
      _pybullet_getJointInfo in pybullet.c.o
      _pybullet_getVisualShapeData in pybullet.c.o
  "_Py_InitModule4_64", referenced from:
      _initpybullet in pybullet.c.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [examples/pybullet/pybullet.2.87.dylib] Error 1
make[1]: *** [examples/pybullet/CMakeFiles/pybullet.dir/all] Error 2
make: *** [all] Error 2

#1106 is also a related issue.
@ahundt ahundt mentioned this issue May 9, 2017
Closed
4 tasks
@erwincoumans
Copy link
Member

erwincoumans commented May 9, 2017 via email

@ahundt ahundt mentioned this issue May 9, 2017
Closed
@ahundt
Copy link
Author

ahundt commented May 9, 2017 •
edited
Loading

As mentioned in #1106 I use ROS and TensorFlow, which means using both the C/C++ & Python APIs, and I was hoping to use a single installation for integrating both. Wouldn't that be best handled via the CMake install process?

@erwincoumans
Copy link
Member

Isn't the python 2 on Mac OSX just 32bit? Your cmake seems to compiles in 64bit mode. Could that be the cause of the linker errors?

@ahundt
Copy link
Author

ahundt commented May 10, 2017 •
edited
Loading

It is the 64 bit python installed by homebrew:

± brew info python
python: stable 2.7.13 (bottled), HEAD
Interpreted, interactive, object-oriented programming language
https://www.python.org
/usr/local/Cellar/python/2.7.13 (3,630 files, 50.2MB) *
  Poured from bottle on 2017-01-09 at 22:20:08
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/python.rb

± which python
/usr/local/bin/python

± python -c 'import struct;print( 8 * struct.calcsize("P"))'
64

If I remove the manually specified python include/lib paths the cmake find script does appear to mix and match finding python2 and python3 headers/libraries. For that item, perhaps incorporating a more robust cmake implementation that finds python could be the way to go?

https://github.com/opencv/opencv/blob/dd379ec9fddc1a1886766cf85844a6e18d38c4f1/cmake/OpenCVDetectPython.cmake

@erwincoumans
Copy link
Member

Well, I think pip install pybullet is the way to go. Perhaps it is better to remove cmake support to build pybullet, since even if you build pybullet, you still have to make sure python finds the module. That is exactly what pip does. You can also run python setup.py install in the root of Bullet. I wouldn't waste time trying to get cmake building pybullet.

@erwincoumans
Copy link
Member

If you run the script ./build_cmake_pybullet_double.sh you should get pybullet build.
If not, what error(s) do you see? Ignore the "pybullet.so" exists warning.

@ahundt
Copy link
Author

ahundt commented May 14, 2017

Okay pip is working well enough, I'll close this. Thanks!

@ahundt ahundt closed this as completed May 14, 2017
@erwincoumans
Copy link
Member

Well, I'm still curious why ./build_cmake_pybullet_double.sh fails for you.
Can you report the result? It should really build pybullet on Mac OSX and Linux just fine.

@ahundt ahundt reopened this May 14, 2017
@ahundt
Copy link
Author

ahundt commented May 14, 2017

Sure here is the output, I deleted the build_cmake directory then I ran ./build_cmake_pybullet_double.sh twice so successfully compiled files don't clutter things.

First run cmake output (detects incorrect python):

± ./build_cmake_pybullet_double.sh
rm: CMakeCache.txt: No such file or directory
-- The C compiler identification is AppleClang 8.0.0.8000042
-- The CXX compiler identification is AppleClang 8.0.0.8000042
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found OpenGL: /System/Library/Frameworks/OpenGL.framework
OPENGL FOUND
/System/Library/Frameworks/OpenGL.framework
-- Found PythonLibs: /usr/local/Frameworks/Python.framework/Versions/3.6/lib/libpython3.6m.dylib (found version "3.6.0")
Mac OSX Version is 10.11
/System/Library/Frameworks/Cocoa.framework
/System/Library/Frameworks/Cocoa.framework
/System/Library/Frameworks/Cocoa.framework
/System/Library/Frameworks/Cocoa.framework
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/athundt/source/bullet3/build_cmake

second run output, linking failed:

± ./build_cmake_pybullet_double.sh
rm: CMakeCache.txt: No such file or directory
mkdir: build_cmake: File exists
OPENGL FOUND
/System/Library/Frameworks/OpenGL.framework
Mac OSX Version is 10.11
/System/Library/Frameworks/Cocoa.framework
/System/Library/Frameworks/Cocoa.framework
/System/Library/Frameworks/Cocoa.framework
/System/Library/Frameworks/Cocoa.framework
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/athundt/source/bullet3/build_cmake
[  2%] Built target Bullet3Common
[  2%] Built target gtest
[  2%] Built target BussIK
[  3%] Built target clsocket
[  3%] Built target LinearMath
[  4%] Built target HACD
[  5%] Built target Bullet3Geometry
[  5%] Built target Bullet2FileLoader
[  6%] Built target App_obj2sdf
[  8%] Built target OpenGLWindow
[ 14%] Built target BulletFileLoader
[ 17%] Built target gwen
[ 17%] Built target BulletInverseDynamics
[ 18%] Built target Test_Collision
[ 19%] Built target Bullet3Collision
[ 30%] Built target BulletCollision
[ 30%] Built target Bullet3Dynamics
[ 32%] Built target ConvexDecomposition
[ 39%] Built target BulletDynamics
[ 40%] Built target Bullet3OpenCL_clew
[ 40%] Built target GIMPACTUtils
[ 40%] Built target App_HelloWorld
[ 41%] Built target App_BasicExample
[ 42%] Built target Test_btKinematicCharacterController
[ 42%] Built target Test_BulletDynamics
[ 42%] Built target BulletWorldImporter
[ 43%] Built target BulletSoftBody
[ 43%] Built target AppBasicExampleGui
[ 45%] Built target BulletInverseDynamicsUtils
[ 46%] Built target Test_BulletInverseDynamics
[ 46%] Built target Test_BulletInverseDynamicsJacobian
[ 47%] Built target BulletXmlWorldImporter
[ 49%] Built target BulletExampleBrowserLib
[ 52%] Built target Test_BulletInverseForwardDynamics
[ 57%] Built target Test_PhysicsClientServer
[ 63%] Built target App_PhysicsServer_SharedMemory
[ 70%] Built target App_PhysicsServer_SharedMemory_GUI
[ 71%] Linking CXX shared library pybullet.dylib
[ 78%] Built target App_RobotSimulator
Undefined symbols for architecture x86_64:
  "_PyInt_FromLong", referenced from:
      _pybullet_connectPhysicsServer in pybullet.o
      _pybullet_loadSDF in pybullet.o
      _pybullet_loadBullet in pybullet.o
      _pybullet_loadMJCF in pybullet.o
      _pybullet_getNumBodies in pybullet.o
      _pybullet_getBodyUniqueId in pybullet.o
      _pybullet_getNumConstraints in pybullet.o
      ...
  "_PyString_FromString", referenced from:
      _pybullet_getBodyInfo in pybullet.o
      _pybullet_getJointInfo in pybullet.o
      _pybullet_getVisualShapeData in pybullet.o
  "_Py_InitModule4_64", referenced from:
      _initpybullet in pybullet.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [examples/pybullet/pybullet.2.87.dylib] Error 1
make[1]: *** [examples/pybullet/CMakeFiles/pybullet.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 93%] Built target App_ExampleBrowser
make: *** [all] Error 2
ln: pybullet.so: File exists

@ahundt
Copy link
Author

ahundt commented May 20, 2017

Okay I've started narrowing this down. I believe the problem is due to bugs in the python find scripts.

For instance with default settings from the pybullet cmake shell script I get the following error:


[ 71%] Linking CXX shared library pybullet.dylib
Undefined symbols for architecture x86_64:
  "_PyInt_FromLong", referenced from:
      _pybullet_connectPhysicsServer in pybullet.o
      _pybullet_loadSDF in pybullet.o
      _pybullet_loadBullet in pybullet.o
      _pybullet_loadMJCF in pybullet.o
      _pybullet_getNumBodies in pybullet.o
      _pybullet_getBodyUniqueId in pybullet.o
      _pybullet_getNumConstraints in pybullet.o
      ...
  "_PyString_FromString", referenced from:
      _pybullet_getBodyInfo in pybullet.o
      _pybullet_getJointInfo in pybullet.o
      _pybullet_getVisualShapeData in pybullet.o
  "_Py_InitModule4_64", referenced from:
      _initpybullet in pybullet.o

I added message(AUTHOR_WARNING ${PYTHON_LIBRARIES}) to the CMakelist.txt in the same folder as pybullet.c and got the following:

CMake Warning (dev) at examples/pybullet/CMakeLists.txt:159 (message):

  /usr/local/Frameworks/Python.framework/Versions/3.6/lib/libpython3.6m.dylib
This warning is for project developers.  Use -Wno-dev to suppress it.

However, I'm running python 2.7 as installed by homebrew.

± which python
/usr/local/bin/python

So, essentially the CMake find script has some bugs, I'll see if I can find a replacement version and put it in.

@erwincoumans
Copy link
Member

Thanks for looking into this, it would be good to have this working.

@ahundt ahundt mentioned this issue May 21, 2017
Merged
ahundt added a commit to ahundt/Pangolin that referenced this issue May 21, 2017
@ahundt
Python remains optional with these changes.
ec7e972 
A specific python version can now be selected by setting PYTHON_VERSION_PANGOLIN.
The python library version must now match the interpreter version exactly.
If all required python dependencies are found, pangolin python utilities are now enabled by default.
Changes incorporate the following BSD licensed cmake code:

https://github.com/BVLC/caffe/blob/32bf5c7ad804ad683aa5ea9382209e9284451e5f/CMakeScripts/FindNumPy.cmake
https://github.com/NikolausDemmel/CMake/pull/2/files
https://github.com/Kitware/CMake/blob/86578eccf2e82286248796bad1032cd0e3a5e1e2/Modules/SelectLibraryConfigurations.cmake

Based on my pull request for the same python cmake issues in pybullet:
bulletphysics/bullet3#1131
bulletphysics/bullet3#1107
@ahundt ahundt mentioned this issue May 21, 2017
Closed
@bulletphysics bulletphysics locked and limited conversation to collaborators Apr 26, 2021

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

Assignees
No one assigned
Labels
new feature
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ahundt @erwincoumans