Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

macOS Sierra python3.5.1 Library not loaded: @rpath/libopenblasp-r0.2.19.dylib #69

Closed
terencezl opened this issue Apr 19, 2017 · 24 comments

Comments

@terencezl
Copy link

terencezl commented Apr 19, 2017

Hi. I encountered this after installing opencv with conda install -c conda-forge opencv.

>>> import cv2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: dlopen(/Applications/anaconda3/envs/data/lib/python3.5/site-packages/cv2.cpython-35m-darwin.so, 2): Library not loaded: @rpath/libopenblasp-r0.2.19.dylib
  Referenced from: /Applications/anaconda3/envs/data/lib/libopencv_hdf.3.2.0.dylib
  Reason: image not found

This is regardless of installing blas, libfortran, openblas, dlib and [blas_openblas] builds of numpy, scipy and scikit-learn.

conda info:

               platform : osx-64
          conda version : 4.3.16
       conda is private : False
      conda-env version : 4.3.16
    conda-build version : not installed
         python version : 3.6.0.final.0
       requests version : 2.12.4
       root environment : /Applications/anaconda3  (writable)
    default environment : /Applications/anaconda3/envs/data
       envs directories : /Applications/anaconda3/envs
                          /Users/[USER]/.conda/envs
          package cache : /Applications/anaconda3/pkgs
                          /Users/[USER]/.conda/pkgs
           channel URLs : https://conda.anaconda.org/anaconda-fusion/osx-64
                          https://conda.anaconda.org/anaconda-fusion/noarch
                          https://repo.continuum.io/pkgs/free/osx-64
                          https://repo.continuum.io/pkgs/free/noarch
                          https://repo.continuum.io/pkgs/r/osx-64
                          https://repo.continuum.io/pkgs/r/noarch
                          https://repo.continuum.io/pkgs/pro/osx-64
                          https://repo.continuum.io/pkgs/pro/noarch
            config file : /Users/[USER]/.condarc
           offline mode : False
             user-agent : conda/4.3.16 requests/2.12.4 CPython/3.6.0 Darwin/16.5.0 OSX/10.12.4
                UID:GID : 503:20

I installed it in an environment with python3.5. conda list:

# packages in environment at /Applications/anaconda3/envs/data:
#
blas                      1.1                    openblas    conda-forge
boost                     1.61.0                   py35_0
cairo                     1.14.6                        4    conda-forge
dlib                      19.0            np111py35_blas_openblas_200  [blas_openblas]  conda-forge
fontconfig                2.12.1                        4    conda-forge
freetype                  2.7                           1    conda-forge
gettext                   0.19.7                        1    conda-forge
giflib                    5.1.4                         0    conda-forge
glib                      2.51.4                        0    conda-forge
harfbuzz                  1.3.4                         2    conda-forge
hdf5                      1.8.17                        9    conda-forge
icu                       58.1                          1    conda-forge
jasper                    1.900.1                       4    conda-forge
jpeg                      9b                            0    conda-forge
libffi                    3.2.1                         3    conda-forge
libgfortran               3.0.0                         0    conda-forge
libiconv                  1.14                          4    conda-forge
libpng                    1.6.28                        0    conda-forge
libtiff                   4.0.6                         7    conda-forge
libwebp                   0.5.2                         7    conda-forge
libxml2                   2.9.4                         4    conda-forge
numpy                     1.11.2          py35_blas_openblas_201  [blas_openblas]  conda-forge
openblas                  0.2.18                        6    conda-forge
opencv                    3.2.0               np111py35_0    conda-forge
openssl                   1.0.2k                        1
pcre                      8.39                          0    conda-forge
pip                       9.0.1                    py35_1
pixman                    0.34.0                        0    conda-forge
python                    3.5.3                         1
readline                  6.2                           2
setuptools                27.2.0                   py35_0
sqlite                    3.13.0                        0
tk                        8.5.18                        0
wheel                     0.29.0                   py35_0
xz                        5.2.2                         1
zlib                      1.2.8                         3

I tried menpo's opencv3 build and there was no dependency issue.

@jakirkham
Copy link
Member

Please include conda list for the relevant environment. Also please include conda info. Can edit the OP to include these.

@jakirkham
Copy link
Member

jakirkham commented Apr 19, 2017

That's weird. Not sure why openblas 0.2.18 is in there. It wouldn't be compatible with this copy of opencv and would not satisfy the version requirements. Can you please try the command below?

conda install openblas=0.2.19

@terencezl
Copy link
Author

UnsatisfiableError: The following specifications were found to be in conflict:
  - dlib -> openblas 0.2.18|0.2.18.*.
  - openblas 0.2.19*
Use "conda info <package>" to see the dependencies for each package.

@jakirkham
Copy link
Member

I see. Still not sure sure how we got opencv installed in this case given they are incompatible.

Basically looks like an issue with dlib. Seems like PR ( conda-forge/dlib-feedstock#1 ) is needed. Though that PR is pretty old and needs some fixing.

If you have time, might be worth taking a look. Would need to re-render the feedstock and update of the pinnings based on our current listing.

@terencezl
Copy link
Author

terencezl commented Apr 19, 2017

This dependency issue occurred even before I installed dlib. I removed dlib and reinstalled openblas 0.2.19-1, and kept opencv-3.2.0-np111py35_0.

The following packages will be UPDATED:

    numpy:    1.11.2-py35_blas_openblas_201 conda-forge [blas_openblas] --> 1.11.3-py35_0
    openblas: 0.2.18-6                      conda-forge --> 0.2.19-1

The problem still existed.

Then I removed opencv and reinstalled it, this time an openblas version 3.2.0-np112py35_blas_openblas_201 popped up, probably because I already installed blas and openblas.

The following NEW packages will be INSTALLED:

    bzip2:  1.0.6-1                           conda-forge
    ffmpeg: 2.8.6-5                           conda-forge
    opencv: 3.2.0-np112py35_blas_openblas_201 conda-forge [blas_openblas]
    x264:   20131217-3                        conda-forge

The following packages will be UPDATED:

    numpy:  1.11.3-py35_0                                 --> 1.12.1-py35_blas_openblas_200 conda-forge [blas_openblas]

The problem was still the same though, asking for @rpath/libopenblasp-r0.2.19.dylib, even when I already installed it. I tried removing blas, openblas and opencv, and reinstalling opencv 3.2.0-np112py35_0, but problem persisted.

Looks like

  1. openblas only created libopenblas.a, libopenblas_nehalemp-r0.2.19.a, libopenblas.dylib, libopenblas_nehalemp-r0.2.19.dylib. Not libopenblasp-r0.2.19.dylib.
  2. All different versions of opencv, not only the [openblas] version, require libopenblasp-r0.2.19.dylib.

@jakirkham
Copy link
Member

Yeah, one needs to use the latest build of opencv. Going back to build number 0 or 1 of opencv 3.2.0 is just going to cause a problem with openblas. Hence why it was rebuilt with the build number 200 and increased from there. To avoid further confusion, I have pulled the old builds of opencv 3.2.0 that are known to be problematic.

Also not sure how libopenblasp-r0.2.19.dylib is missing from openblas 0.2.19. All Mac build from conda-forge for openblas 0.2.19 have that library. Just now verified this. Something else is happening, but I'm at a loss debugging this with the information provided.

As far as getting something working, was able to get a working environment locally without issue and have just tested it. Did nothing more than create a fresh environment with opencv, openblas, and python=3.5. Not entirely sure why that is not working in your case beyond the dlib issue. However have included an environment file below that you should be able to use without issue. Would recommend creating a new environment with that. These docs should help. Then maybe try slowly adding in other components.

# environment.yml
name: opencv
channels: !!python/tuple
- !!python/unicode
  'conda-forge'
- !!python/unicode
  'defaults'
dependencies:
- conda-forge::blas=1.1=openblas
- conda-forge::bzip2=1.0.6=1
- conda-forge::ca-certificates=2017.1.23=0
- conda-forge::cairo=1.14.6=4
- conda-forge::ffmpeg=2.8.6=5
- conda-forge::fontconfig=2.12.1=4
- conda-forge::freetype=2.7=1
- conda-forge::gettext=0.19.7=1
- conda-forge::giflib=5.1.4=0
- conda-forge::glib=2.51.4=0
- conda-forge::harfbuzz=1.3.4=2
- conda-forge::hdf5=1.8.17=9
- conda-forge::icu=58.1=1
- conda-forge::jasper=1.900.1=4
- conda-forge::jpeg=9b=0
- conda-forge::libffi=3.2.1=3
- conda-forge::libgfortran=3.0.0=0
- conda-forge::libiconv=1.14=4
- conda-forge::libpng=1.6.28=0
- conda-forge::libtiff=4.0.6=7
- conda-forge::libwebp=0.5.2=7
- conda-forge::libxml2=2.9.4=4
- conda-forge::ncurses=5.9=10
- conda-forge::numpy=1.12.1=py35_blas_openblas_200
- conda-forge::openblas=0.2.19=2
- conda-forge::opencv=3.2.0=np112py35_blas_openblas_201
- conda-forge::openssl=1.0.2k=0
- conda-forge::pcre=8.39=0
- conda-forge::pixman=0.34.0=0
- conda-forge::python=3.5.3=1
- conda-forge::readline=6.2=0
- conda-forge::sqlite=3.13.0=1
- conda-forge::tk=8.5.19=1
- conda-forge::x264=20131217=3
- conda-forge::xz=5.2.2=0
- conda-forge::zlib=1.2.11=0

@terencezl
Copy link
Author

terencezl commented Apr 19, 2017

I see now.

All Mac build from conda-forge for openblas 0.2.19 have that library.

I installed the default cloud's openblas 0.2.19 and that only has libopenblas.a, libopenblas_nehalemp-r0.2.19.a, libopenblas.dylib, libopenblas_nehalemp-r0.2.19.dylib. Not libopenblasp-r0.2.19.dylib. When I was asked to conda install openblas=0.2.19 I did not have conda-forge added as a channel.

The builds opencv-3.2.0-np111py35_0, and 3.2.0-np112py35_blas_openblas_201 all in effect depend on conda-forge's openblas. Should it happen though? Shouldn't only 3.2.0-np112py35_blas_openblas_201 depend on it?

I think what caused this is partly opencv not asking for openblas as a dep, and further, not asking for conda-forge's build because again, the default build doesn't have that file. I'm not an expert on this. I'm still confused why blas is installed separately depending on openblas... What's the difference...

Then comes in dlib, which requires me downgrading opencv and openblas.

> conda install -c conda-forge dlib
The following NEW packages will be INSTALLED:

    blas:     1.1-openblas                     conda-forge
    dlib:     19.0-np111py35_blas_openblas_200 conda-forge [blas_openblas]

The following packages will be SUPERSEDED by a higher-priority channel:

    numpy:    1.12.1-py35_0                                --> 1.11.2-py35_blas_openblas_201 conda-forge [blas_openblas]
    opencv:   3.2.0-np112py35_0                conda-forge --> 3.1.0-np111py35_1             conda-forge

The following packages will be DOWNGRADED due to dependency conflicts:

    openblas: 0.2.19-2                         conda-forge --> 0.2.18-6                      conda-forge

@terencezl
Copy link
Author

terencezl commented Apr 19, 2017

I'm still confused why blas is installed separately depending on openblas... What's the difference...

This was somewhat answered in conda-forge/conda-forge.github.io#232.

@patricksnape
Copy link
Contributor

opencv-3.2.0-np111py35_0 This is a broken build - OpenCV depends on openblas - it incorrectly didn't before and that was a bug. Please use the openblas build.

@jakirkham
Copy link
Member

The blas package was created to allow selection of different BLASes in the stack. So if we one day used BLIS and/or ATLAS in addition to OpenBLAS, one could pick which one they would like to use independent of picking other packages that require a BLAS (e.g. numpy).

In any event, this really isn't relevant to the issue at hand other than some builds of OpenCV linked to the BLAS used and we didn't properly express that in the package as we weren't aware of this change.

@jakirkham
Copy link
Member

TL;DR there were some bad opencv packages that didn't express their dependencies correctly. We fixed the issue in new packages and have now pulled the bad packages. So this should be resolved. Please let us know if there are issues with the new packages or if the bad packages come back for some reason.

@terencezl
Copy link
Author

So far so good! Good job and thanks!

@fcr
Copy link

fcr commented May 11, 2017

Python 3.6. Sierra 10.12.4, conda 4.3.17
On my last conda update --all I was offered:
The following packages will be UPDATED:

bzip2:        1.0.6-1                            conda-forge --> 1.0.6-3
gettext:      0.19.7-1                           conda-forge --> 0.19.8-1
libgfortran:  3.0.0-0                            conda-forge --> 3.0.0-1
pcre:         8.39-0                             conda-forge --> 8.39-1
tornado:      4.4.2-py36_0                                   --> 4.5.1-py36_0

The following packages will be SUPERSEDED by a higher-priority channel:

expat:        2.1.0-2                            conda-forge --> 2.1.0-0
libffi:       3.2.1-3                            conda-forge --> 3.2.1-1
mock:         2.0.0-py36_0                       conda-forge --> 2.0.0-py36_0
numpy:        1.12.1-py36_blas_openblas_200      conda-forge [blas_openblas] --> 1.12.1-py36_0
openblas:     0.2.19-2                           conda-forge --> 0.2.19-1
pbr:          3.0.0-py36_0                       conda-forge --> 1.10.0-py36_0
pixman:       0.34.0-0                           conda-forge --> 0.34.0-0
scikit-learn: 0.18.1-np112py36_blas_openblas_200 conda-forge [blas_openblas] --> 0.18.1-np112py36_1
scipy:        0.19.0-np112py36_blas_openblas_200 conda-forge [blas_openblas] --> 0.19.0-np112py36_0

When I allowed this and tried to import cv2 I got the following error:
ImportError: dlopen(/Users/fred/anaconda/envs/python3test/lib/python3.6/site-packages/cv2.cpython-36m-darwin.so, 2): Library not loaded: @rpath/libopenblasp-r0.2.19.dylib
Referenced from: /Users/fred/anaconda/envs/python3test/lib/libopencv_hdf.3.2.0.dylib
Reason: image not found

Checking the lib folder there was a different openblas library.
Unloading and loading opencv did not restore the original openblas library but the install did not complain.

@terencezl
Copy link
Author

@fcr when you update make sure switch on -c conda-forge.

@fcr
Copy link

fcr commented May 11, 2017

Thanks @terencezl
I tried it with:

conda update --all -c conda-forge

This updated all my packages to conda-forge.
However, when now when I try to import cv2 I get the error:

ImportError: dlopen(/Users/fred/anaconda/envs/python3test/lib/python3.6/site-packages/cv2.cpython-36m-darwin.so, 2): Library not loaded: @rpath/libintl.8.dylib Referenced from: /Users/fred/anaconda/envs/python3test/lib/libharfbuzz.0.dylib Reason: image not found

BTW, in principle if one installs even one package from conda-forge does this mean that one has to use the -c conda-forge from there on?

@fcr
Copy link

fcr commented May 11, 2017

I just tried the following and it solved the problem, sort off:

(python3test) ~#conda install harfbuzz -c conda-forge
Fetching package metadata .............
Solving package specifications: .

Package plan for installation in environment /Users/fred/anaconda/envs/python3test:

The following NEW packages will be INSTALLED:

graphite2: 1.3.9-0                           conda-forge

The following packages will be UPDATED:

harfbuzz:  1.3.4-2                           conda-forge --> 1.4.3-0           conda-forge

The following packages will be DOWNGRADED due to dependency conflicts:

opencv:    3.2.0-np112py36_blas_openblas_201 conda-forge [blas_openblas] --> 3.1.0-np112py36_1 conda-forge

But left me with an older version of opencv

@patricksnape
Copy link
Contributor

That version of harfbuzz is not in the list of pinned packages. You need harfbuzz 1.3.*.

@fcr
Copy link

fcr commented May 14, 2017

Thanks.

@EchoChung6
Copy link

Hi Terence, i came across the same issue as you did.
Firstly thank you for sharing information here. However it is too informative hence i did not extract the solution which you actually solve this issue. Could you kindly let me know how you fixed it and can you now successfully import cv2 in anaconda? Many thanks

@EchoChung6
Copy link

Hi @jakirkham I came across the same issue, yet it seems alright with my conda list:
openblas 0.2.19 2 conda-forge
opencv 3.2.0 np111py27_blas_openblas_201 [blas_openblas] conda-forge
openpyxl 2.4.1 py27_0

@patricksnape
Copy link
Contributor

Please make a new environment and install only OpenCV in it and see if that works conda install -c conda-forge opencv --override-channels

@ChrissPavel
Copy link

@jakirkham Thank you, also worked using your installation by hand with your proposed environment file environment.yml

@yatianqu-purigen
Copy link

Thank you very much for the post!
I had the same problem. It turns out the version of openblas was not right. It was openblas 0.2.1.

I tried:
conda install openblas=0.2.19

It solved the problem.

@williamyts123
Copy link

Hi Guys, I have a similar problem and get stuck here long.
I try to import sklearn but has the following error

ImportError: dlopen(/Users/Monday123/Downloads/anaconda/lib/python3.6/site-packages/daal/_daal.cpython-36m-darwin.so, 2): Library not loaded: @rpath/libdaal_core.dylib
Referenced from: /Users/Monday123/Downloads/anaconda/lib/python3.6/site-packages/daal/_daal.cpython-36m-darwin.so
Reason: image not found

I did check daal package is in the folder but by some reason has this issue.

Many Thanks

@conda-forge conda-forge locked as resolved and limited conversation to collaborators Oct 9, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants