Skip to content
This repository has been archived by the owner on Apr 7, 2022. It is now read-only.

Generalise search for libboost_python on POSIX #335

Merged
merged 5 commits into from
Mar 9, 2020

Conversation

ajoubertza
Copy link
Member

@ajoubertza ajoubertza commented Mar 1, 2020

The naming of the boost library has changed around version 1.67, so the current approach in setup.py was failing on newer Linux distributions. Instead of hardcoding the suffix for different
distributions, use the ctypes.utils.find_library function and try the various permutations. The find_library function uses ldconfig and a few other tools under the hood, so it should be reliable.

A new environment variable option was added for the case where the search fails. The library name can be specified directly by setting BOOST_LIB.

Fixes: #300 and #310

Also remove the reporting of boost version from runtime info since we don't get the boost version at runtime - displaying "0.0.0" leads to confusion.

For interest, here's a review of the current Python boost library names:

Path: /usr/lib/x86_64-linux-gnu/, except for CentOS in /usr/lib64/

Distribution Release Boost version Files
Ubuntu 14.04 (Trusty) 1.54 & 1.55 libboost_python-py27.a
libboost_python-py27.so
libboost_python-py33.so
libboost_python-py34.so
libboost_python.so
Ubuntu 16.04 (Xenial) 1.58 libboost_python-py27.so
libboost_python-py35.so
libboost_python.so
Ubuntu 18.04 (Bionic) 1.65 libboost_python-py27.so
libboost_python-py36.so
libboost_python.so
libboost_python3-py36.so
libboost_python3.so
Ubuntu 19.04 (Disco) 1.67 libboost_python.so
libboost_python27.so
libboost_python3-py37.so
libboost_python3.so
libboost_python37.so
Ubuntu 20.04 (Focal) 1.71 ? libboost_python37.so
libboost_python38.so
Debian 8 (Jessie) 1.55 libboost_python-py27.so
libboost_python-py34.so
libboost_python.so
Debian 9 (Stretch) 1.62 libboost_python-py27.so
libboost_python-py35.so
libboost_python.so
Debian 10 (Buster) 1.67 libboost_python.so
libboost_python27.so
libboost_python3-py37.so
libboost_python3.so
libboost_python37.so
Debian 11 (Bullseye) 1.67 ? libboost_python.so
libboost_python27.so
libboost_python3-py37.so
libboost_python3.so
libboost_python37.so
libboost_python38.so
Debian Experimental (Sid) 1.71 ? libboost_python37.so
libboost_python38.so
CentOS 6 1.41 libboost_python-mt.so
libboost_python.so
CentOS 7 1.53 libboost_python3-mt.so
libboost_python3.so
libboost_python3-mt.so
libboost_python3.so
CentOS 8 1.66 libboost_python3.so
Gentoo 2.6 1.72 libboost_python27.so
libboost_python36.so

We don't get the boost version at runtime, so don't display anything,
as it leads to confusion.

If it is every added in future, this is the related Stack Overflow
post:  https://stackoverflow.com/questions/20140321/how-to-find-boost-runtime-version
The naming of the boost library has changed around version 1.67,
so the current approach in `setup.py` was failing on newer Linux
distributions.  Instead of hardcoding the suffix for different
distributions, use the `ctypes.utils.find_library` function and try
the various permutations.  The `find_library` function uses `ldconfig`
and a few other tools under the hood.

A new environment variable option was added for the case where
the search fails.  The library name can be specified directly by
setting `BOOST_LIB`.
@stanislaw55
Copy link

Hi! I think this is great addition to installation process! I run tests on CentOS 7

Results of tests:
CentOS 7, Boost 1.48, Python 2.7: FAIL (headers are located in /usr/include/boost148 and are not picked by compiler)
CentOS 7, Boost 1.53, Python 2.7: PASS
CentOS 7, Boost 1.69, Python 2.7: FAIL (headers are located in /usr/include/boost169 and are not picked by compiler)
CentOS 7, Boost 1.48, Python 3.6: Could not test, no library for Python 3
CentOS 7, Boost 1.53, Python 3.6: PASS
CentOS 7, Boost 1.69, Python 3.6: FAIL (headers are located in /usr/include/boost169 and are not picked by compiler)

I think there should be also added variable like BOOST_HEADERS to point to proper headers directory

@ajoubertza
Copy link
Member Author

@stanislaw55 Thanks for testing. So the standard CentOS install works fine, but not custom.

Can you not use the BOOST_ROOT environment variable, and put your whole custom boost installation in to separate folder? In the example below, we would set BOOST_ROOT=/opt/my_boost:

/opt# tree
.
└── my_boost
    ├── include
    └── lib64

If not, the we can add BOOST_HEADERS, but how to combine BOOST_ROOT and BOOST_HEADERS if both are provided?

Allow for more custom installation options.
@ajoubertza
Copy link
Member Author

@stanislaw55 I've added more options now - please have a look and can you test?

@ajoubertza
Copy link
Member Author

@stanislaw55 I tried out the new environment vars, moving my boost header and libraries around. It works correctly for me. Note: For the headers, the C++ extension code uses includes like
#include <boost/python.hpp>, so the boost headers have to be in a directory called boost.

In your example of /usr/include/boost169, there needs to be a boost directory inside, so there would be a file like /usr/include/boost169/boost/python.hpp.

@stanislaw55
Copy link

Hi,
I performed tests again and here are the results

Results of tests:
CentOS 7, Boost 1.48, Python 2.7: PASS (BOOST_HEADERS=/usr/include/boost148)
CentOS 7, Boost 1.53, Python 2.7: PASS
CentOS 7, Boost 1.69, Python 2.7: PASS (BOOST_HEADERS=/usr/include/boost169)
CentOS 7, Boost 1.48, Python 3.6: Could not test, no library for Python 3
CentOS 7, Boost 1.53, Python 3.6: PASS
CentOS 7, Boost 1.69, Python 3.6: PASS (BOOST_HEADERS=/usr/include/boost169)

Looks like it now is possible to compile PyTango on CentOS 7 without hassle!

@ajoubertza
Copy link
Member Author

Looks like it now is possible to compile PyTango on CentOS 7 without hassle!

Excellent. Thanks for the thorough tests.

@ajoubertza ajoubertza merged commit e56888e into develop Mar 9, 2020
@ajoubertza ajoubertza deleted the issue-300-310-libboost branch March 9, 2020 19:56
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

boost-python lib name mismatch on current debian
2 participants