ImportError: cannot import name '_C' from 'pytorch3d' (Installation Error)

[ParthGoyal1508]

Installation issue from a local clone. I installed as mentioned here.

The build was successful but got errors in execution.

>>> import torch
>>> import pytorch3d
>>> from pytorch3d import _C
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name '_C' from 'pytorch3d' (/home/srijan.singh/pytorch3d/pytorch3d/__init__.py)

Instructions To Reproduce the Issue:

1. Any changes you made (git diff) or code you wrote

No change 

2. The exact command(s) you ran:

conda create -n Test 
conda activate Test
conda install -c pytorch pytorch=1.9.1 torchvision cudatoolkit=10.2
conda install -c fvcore -c iopath -c conda-forge fvcore iopath
conda install -c bottler nvidiacub

git clone https://github.com/facebookresearch/pytorch3d.git
cd pytorch3d 
python3 setup.py install 

3. What you observed (including the full logs):

Traceback (most recent call last):
  File "/home/srijan.singh/pytorch3d/test.py", line 1, in <module>
    from pytorch3d.loss import chamfer_distance
  File "/home/srijan.singh/pytorch3d/pytorch3d/loss/__init__.py", line 8, in <module>
    from .chamfer import chamfer_distance
  File "/home/srijan.singh/pytorch3d/pytorch3d/loss/chamfer.py", line 11, in <module>
    from pytorch3d.ops.knn import knn_gather, knn_points
  File "/home/srijan.singh/pytorch3d/pytorch3d/ops/__init__.py", line 7, in <module>
    from .ball_query import ball_query
  File "/home/srijan.singh/pytorch3d/pytorch3d/ops/ball_query.py", line 10, in <module>
    from pytorch3d import _C
ImportError: cannot import name '_C' from 'pytorch3d' (/home/srijan.singh/pytorch3d/pytorch3d/__init__.py)

4. Output of conda list

# packages in environment at /home/srijan.singh/miniconda3/envs/Test:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main  
_openmp_mutex             4.5                       1_gnu  
blas                      1.0                         mkl  
ca-certificates           2021.10.26           h06a4308_2  
certifi                   2021.10.8        py39h06a4308_0  
colorama                  0.4.4              pyh9f0ad1d_0    conda-forge
cudatoolkit               10.2.89              hfd86e86_1  
freetype                  2.11.0               h70c0345_0  
fvcore                    0.1.5.post20210804    pyhd8ed1ab_0    conda-forge
giflib                    5.2.1                h7b6447c_0  
intel-openmp              2021.4.0          h06a4308_3561  
iopath                    0.1.9                      py39    iopath
jpeg                      9d                   h7f8727e_0  
lcms2                     2.12                 h3be6417_0  
ld_impl_linux-64          2.35.1               h7274673_9  
libffi                    3.3                  he6710b0_2  
libgcc-ng                 9.3.0               h5101ec6_17  
libgomp                   9.3.0               h5101ec6_17  
libpng                    1.6.37               hbc83047_0  
libstdcxx-ng              9.3.0               hd4cf53a_17  
libtiff                   4.2.0                h85742a9_0  
libuv                     1.40.0               h7b6447c_0  
libwebp                   1.2.0                h89dd481_0  
libwebp-base              1.2.0                h27cfd23_0  
lz4-c                     1.9.3                h295c915_1  
mkl                       2021.4.0           h06a4308_640  
mkl-service               2.4.0            py39h7f8727e_0  
mkl_fft                   1.3.1            py39hd3c417c_0  
mkl_random                1.2.2            py39h51133e4_0  
ncurses                   6.3                  h7f8727e_2  
ninja                     1.10.2           py39hd09550d_3  
numpy                     1.21.2           py39h20f2e39_0  
numpy-base                1.21.2           py39h79a1101_0  
nvidiacub                 1.10.0                        0    bottler
olefile                   0.46               pyhd3eb1b0_0  
openssl                   1.1.1l               h7f8727e_0  
pillow                    8.4.0            py39h5aabda8_0  
pip                       21.2.4           py39h06a4308_0  
portalocker               2.3.2            py39hf3d152e_1    conda-forge
python                    3.9.7                h12debd9_1  
python_abi                3.9                      2_cp39    conda-forge
pytorch                   1.9.1           py3.9_cuda10.2_cudnn7.6.5_0    pytorch
pytorch3d                 0.6.1                    pypi_0    pypi
pyyaml                    5.4.1            py39h3811e60_0    conda-forge
readline                  8.1                  h27cfd23_0  
setuptools                58.0.4           py39h06a4308_0  
six                       1.16.0             pyhd3eb1b0_0  
sqlite                    3.37.0               hc218d9a_0  
tabulate                  0.8.9              pyhd8ed1ab_0    conda-forge
termcolor                 1.1.0                      py_2    conda-forge
tk                        8.6.11               h1ccaba5_0  
torchvision               0.2.2                      py_3    pytorch
tqdm                      4.62.3             pyhd8ed1ab_0    conda-forge
typing_extensions         3.10.0.2           pyh06a4308_0  
tzdata                    2021e                hda174b7_0  
wheel                     0.37.0             pyhd3eb1b0_1  
xz                        5.2.5                h7b6447c_0  
yacs                      0.1.8                    pypi_0    pypi
yaml                      0.2.5                h516909a_0    conda-forge
zlib                      1.2.11               h7f8727e_4  
zstd                      1.4.9                haebb681_0  

Could you please help me to resolve this? Thanks in advance.

[bottler]

Something strange has happened. The output of conda list suggests pytorch3d has been installed from pypi. But in your instructions you didn't do that, you installed it from source. Is there a confusion between environments? Can you paste the output of pip list too?

If pytorch3d is the one you built, then I wonder if there's a cuda version mismatch. What's the output of nvcc --version?

[ParthGoyal1508]

@bottler output of pip3 list:

Package                 Version  
----------------------- ---------
absl-py                 0.12.0   
astunparse              1.6.3    
biopython               1.78     
cachetools              4.2.2    
certifi                 2020.12.5
chardet                 4.0.0    
cycler                  0.10.0   
decorator               4.4.2    
funcsigs                1.0.2    
gast                    0.3.3    
google-auth             1.30.0   
google-auth-oauthlib    0.4.4    
google-pasta            0.2.0    
GridDataFormats         0.5.0    
grpcio                  1.37.1   
gsd                     2.4.2    
h5py                    2.10.0   
idna                    2.10     
joblib                  1.0.1    
Keras-Preprocessing     1.1.2    
kiwisolver              1.3.1    
Markdown                3.3.4    
matplotlib              3.3.4    
MDAnalysis              1.1.1    
mmtf-python             1.1.2    
mock                    4.0.3    
msgpack                 1.0.2    
networkx                2.5.1    
numpy                   1.18.5   
oauthlib                3.1.0    
opt-einsum              3.3.0    
Pillow                  8.2.0    
pip                     19.2.3   
protobuf                3.16.0   
pyasn1                  0.4.8    
pyasn1-modules          0.2.8    
pyparsing               2.4.7    
python-dateutil         2.8.1    
requests                2.25.1   
requests-oauthlib       1.3.0    
rsa                     4.7.2    
scipy                   1.4.1    
setuptools              41.2.0   
six                     1.16.0   
tensorboard             2.5.0    
tensorboard-data-server 0.6.1    
tensorboard-plugin-wit  1.8.0    
tensorflow              2.3.2    
tensorflow-estimator    2.3.0    
tensorflow-gpu          2.3.0    
tensorflow-text         2.3.0    
termcolor               1.1.0    
tqdm                    4.61.0   
urllib3                 1.26.4   
Werkzeug                1.0.1    
wheel                   0.36.2   
wrapt                   1.12.1

output of nvcc --version:

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Fri_Feb__8_19:08:17_PST_2019
Cuda compilation tools, release 10.1, V10.1.105

[bottler]

You have CUDA version 10.1 but cudatoolkit 10.2 in the environment.

[ParthGoyal1508]

I have downgraded the CUDA version to 10.1 as mentioned and accordingly changed the compatible pytorch, torchvision versions too but again it's throwing the same error.

Command I ran:

conda create -n TestPytorch3d python=3.8
conda activate TestPytorch3d
conda install pytorch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2 cudatoolkit=10.1 -c pytorch
conda install -c fvcore -c iopath -c conda-forge fvcore iopath
conda install -c bottler nvidiacub

rm -rf build/
rm -rf dist/
rm -rf pytorch3d.egg-info/

python3 setup.py install

Build was successful but getting same error.

Output of conda list for your reference.

# packages in environment at /home/srijan.singh/miniconda3/envs/TestPytorch3d:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main  
_openmp_mutex             4.5                       1_gnu  
_pytorch_select           0.1                       cpu_0  
blas                      1.0                         mkl  
ca-certificates           2021.10.26           h06a4308_2  
certifi                   2021.10.8        py38h578d9bd_1    conda-forge
colorama                  0.4.4              pyh9f0ad1d_0    conda-forge
cudatoolkit               10.1.243             h6bb024c_0  
freetype                  2.11.0               h70c0345_0  
fvcore                    0.1.5.post20210915            py38    fvcore
giflib                    5.2.1                h7b6447c_0  
intel-openmp              2021.4.0          h06a4308_3561  
iopath                    0.1.9                      py38    iopath
jpeg                      9d                   h7f8727e_0  
lcms2                     2.12                 h3be6417_0  
ld_impl_linux-64          2.35.1               h7274673_9  
libffi                    3.3                  he6710b0_2  
libgcc-ng                 9.3.0               h5101ec6_17  
libgomp                   9.3.0               h5101ec6_17  
libpng                    1.6.37               hbc83047_0  
libstdcxx-ng              9.3.0               hd4cf53a_17  
libtiff                   4.2.0                h85742a9_0  
libuv                     1.40.0               h7b6447c_0  
libwebp                   1.2.0                h89dd481_0  
libwebp-base              1.2.0                h27cfd23_0  
lz4-c                     1.9.3                h295c915_1  
mkl                       2021.4.0           h06a4308_640  
mkl-service               2.4.0            py38h7f8727e_0  
mkl_fft                   1.3.1            py38hd3c417c_0  
mkl_random                1.2.2            py38h51133e4_0  
ncurses                   6.3                  h7f8727e_2  
ninja                     1.10.2           py38hd09550d_3  
numpy                     1.21.2           py38h20f2e39_0  
numpy-base                1.21.2           py38h79a1101_0  
nvidiacub                 1.10.0                        0    bottler
olefile                   0.46               pyhd3eb1b0_0  
openssl                   1.1.1l               h7f8727e_0  
pillow                    8.4.0            py38h5aabda8_0  
pip                       21.2.4           py38h06a4308_0  
portalocker               2.3.2            py38h578d9bd_1    condaand pip3 list-forge
python                    3.8.12               h12debd9_0  
python_abi                3.8                      2_cp38    conda-forge
pytorch                   1.7.1           py3.8_cuda10.1.243_cudnn7.6.3_0    pytorch
pytorch3d                 0.6.1                    pypi_0    pypi
pyyaml                    5.4.1            py38h497a2fe_0    conda-forge
readline                  8.1                  h27cfd23_0  
setuptools                58.0.4           py38h06a4308_0  
six                       1.16.0             pyhd3eb1b0_0  
sqlite                    3.37.0               hc218d9a_0  
tabulate                  0.8.9              pyhd8ed1ab_0    conda-forge
termcolor                 1.1.0                      py_2    conda-forge
tk                        8.6.11               h1ccaba5_0  
torchaudio                0.7.2                      py38    pytorch
torchvision               0.8.2           cpu_py38ha229d99_0  
tqdm                      4.62.3             pyhd8ed1ab_0    conda-forge
typing_extensions         3.10.0.2           pyh06a4308_0  
wheel                     0.37.0             pyhd3eb1b0_1  
xz                        5.2.5                h7b6447c_0  
yacs                      0.1.8                    pypi_0    pypi
yaml                      0.2.5                h516909a_0    conda-forge
zlib                      1.2.11               h7f8727e_4  
zstd                      1.4.9                haebb681_0  

[bottler]

Do you get the same error if you import torch before you try to use _C?

[ParthGoyal1508]

No import torch does not throwing any error.

Python 3.8.12 (default, Oct 12 2021, 13:49:34) 
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch 
>>> import pytorch3d
>>> from pytorch3d import _C
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name '_C' from 'pytorch3d' (/home/srijan.singh/pytorch3d/pytorch3d/__init__.py)

[bottler]

Oh. I see the location - I think this is a path issue. You have installed pytorch3d in the conda environment, but you are starting python while inside the source code directory of pytorch3d. This means it is looking for the build artifact in the source tree instead of the pytorch3d tree of the conda environment. Can you move to a different directory and try again, e.g. mkdir a; cd a ; python?

[ParthGoyal1508]

okay okay. Got it. Thanks. It worked.

[sudmat]

Do you get the same error if you import torch before you try to use _C?

Hi bottler, since you mentioned this, I had a similar issue. When I execute "from pytorch3d import _C", it says "ImportError: DLL load failed while importing _C: The specified module could not be found". But if I "import torch" before importing _C, it is fine. Any idea of why this is happening? Thank you.

[zn123]

将pytorch3d 改名再测试_1 、跟conda 下的pytorch3d冲突造成的
Renamed pytorch3d to test _1, caused by conflict with pytorch3d under conda