Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

scotch7.0.2 broke macOS #180

Closed
1 task done
guyer opened this issue Dec 10, 2023 · 7 comments · Fixed by #181
Closed
1 task done

scotch7.0.2 broke macOS #180

guyer opened this issue Dec 10, 2023 · 7 comments · Fixed by #181
Labels
bug

Comments

@guyer
Copy link

guyer commented Dec 10, 2023 •
edited
Loading

Solution to issue cannot be found in the documentation.

  • I checked the documentation.

Issue

Since merging #176, petsc4py raises a linkage error on macOS (both osx-64 and osx-arm64). linux-64 is fine.

$ python -c "from petsc4py import PETSc"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/guyer/mambaforge-arm/envs/scotch_eq_7/lib/python3.12/site-packages/petsc4py/PETSc.py", line 3, in <module>
    PETSc = ImportPETSc(ARCH)
            ^^^^^^^^^^^^^^^^^
  File "/Users/guyer/mambaforge-arm/envs/scotch_eq_7/lib/python3.12/site-packages/petsc4py/lib/__init__.py", line 29, in ImportPETSc
    return Import('petsc4py', 'PETSc', path, arch)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/guyer/mambaforge-arm/envs/scotch_eq_7/lib/python3.12/site-packages/petsc4py/lib/__init__.py", line 95, in Import
    module = import_module(pkg, name, path, arch)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/guyer/mambaforge-arm/envs/scotch_eq_7/lib/python3.12/site-packages/petsc4py/lib/__init__.py", line 71, in import_module
    module = importlib.util.module_from_spec(spec)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ImportError: dlopen(/Users/guyer/mambaforge-arm/envs/scotch_eq_7/lib/python3.12/site-packages/petsc4py/lib/PETSc.cpython-312-darwin.so, 0x0002): symbol not found in flat namespace '_SCOTCH_errorPrint'

Environment created with

conda create --channel conda-forge --name scotch_eq_7 "scotch=7" petsc4py

Installed packages

# packages in environment at /Users/guyer/mambaforge-arm/envs/scotch_eq_7:
#
# Name                    Version                   Build  Channel
bzip2                     1.0.8                h93a5062_5    conda-forge
c-ares                    1.23.0               h93a5062_0    conda-forge
ca-certificates           2023.11.17           hf0a4a13_0    conda-forge
fftw                      3.3.10          mpi_openmpi_haef8dc3_8    conda-forge
gmp                       6.3.0                h965bd2d_0    conda-forge
hdf5                      1.14.3          mpi_openmpi_h20f603a_0    conda-forge
hypre                     2.28.0          mpi_openmpi_haba3941_0    conda-forge
icu                       73.2                 hc8870d7_0    conda-forge
krb5                      1.21.2               h92f50d5_0    conda-forge
libaec                    1.1.2                h13dd4ca_1    conda-forge
libblas                   3.9.0           20_osxarm64_openblas    conda-forge
libcblas                  3.9.0           20_osxarm64_openblas    conda-forge
libcurl                   8.5.0                h2d989ff_0    conda-forge
libcxx                    16.0.6               h4653b0c_0    conda-forge
libedit                   3.1.20191231         hc8eb9b7_2    conda-forge
libev                     4.33                 h93a5062_2    conda-forge
libexpat                  2.5.0                hb7217d7_1    conda-forge
libffi                    3.4.2                h3422bc3_5    conda-forge
libgfortran               5.0.0           13_2_0_hd922786_1    conda-forge
libgfortran5              13.2.0               hf226fd6_1    conda-forge
libhwloc                  2.9.3           default_h4394839_1009    conda-forge
libiconv                  1.17                 he4db4b2_0    conda-forge
liblapack                 3.9.0           20_osxarm64_openblas    conda-forge
libnghttp2                1.58.0               ha4dd798_1    conda-forge
libopenblas               0.3.25          openmp_h6c19121_0    conda-forge
libptscotch               7.0.2                h4ec5e76_0    conda-forge
libscotch                 7.0.2                h80c162b_0    conda-forge
libsqlite                 3.44.2               h091b4b1_0    conda-forge
libssh2                   1.11.0               h7a5bd25_0    conda-forge
libxml2                   2.11.6               h0d0cfa8_0    conda-forge
libzlib                   1.2.13               h53f4e23_5    conda-forge
llvm-openmp               17.0.6               hcd81f8e_0    conda-forge
metis                     5.1.1                h965bd2d_2    conda-forge
mpfr                      4.2.1                h9546428_0    conda-forge
mpi                       1.0                     openmpi    conda-forge
mumps-include             5.2.1               hce30654_13    conda-forge
mumps-mpi                 5.2.1               h00b355f_13    conda-forge
ncurses                   6.4                  h463b476_2    conda-forge
numpy                     1.26.2          py312h5d55045_0    conda-forge
openmpi                   4.1.6              h526c993_101    conda-forge
openssl                   3.2.0                h0d3ecfb_1    conda-forge
parmetis                  4.0.3             h6eb5794_1005    conda-forge
petsc                     3.20.2          real_hccb9628_102    conda-forge
petsc4py                  3.20.1          real_h17b5e51_100    conda-forge
pip                       23.3.1             pyhd8ed1ab_0    conda-forge
ptscotch                  7.0.2                hc1c4572_0    conda-forge
python                    3.12.0          h47c9636_0_cpython    conda-forge
python_abi                3.12                    4_cp312    conda-forge
readline                  8.2                  h92ec313_1    conda-forge
scalapack                 2.2.0                h515df86_1    conda-forge
scotch                    7.0.2                hc1c4572_0    conda-forge
setuptools                68.2.2             pyhd8ed1ab_0    conda-forge
suitesparse               5.10.1               h88be0ae_2    conda-forge
superlu                   5.2.2                hc615359_0    conda-forge
tbb                       2021.11.0            h6aa02a4_0    conda-forge
tk                        8.6.13               h5083fa2_1    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
wheel                     0.42.0             pyhd8ed1ab_0    conda-forge
xz                        5.2.6                h57fd34a_0    conda-forge
yaml                      0.2.5                h3422bc3_2    conda-forge
zlib                      1.2.13               h53f4e23_5    conda-forge
zstd                      1.5.5                h4f39d0f_0    conda-forge

Environment info

active environment : scotch_eq_7
    active env location : /Users/guyer/mambaforge-arm/envs/scotch_eq_7
            shell level : 2
       user config file : /Users/guyer/.condarc
 populated config files : /Users/guyer/mambaforge-arm/.condarc
                          /Users/guyer/.condarc
          conda version : 23.7.4
    conda-build version : not installed
         python version : 3.10.10.final.0
       virtual packages : __archspec=1=arm64
                          __osx=13.6.2=0
                          __unix=0=0
       base environment : /Users/guyer/mambaforge-arm  (writable)
      conda av data dir : /Users/guyer/mambaforge-arm/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/osx-arm64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/osx-arm64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/osx-arm64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /Users/guyer/mambaforge-arm/pkgs
                          /Users/guyer/.conda/pkgs
       envs directories : /Users/guyer/mambaforge-arm/envs
                          /Users/guyer/.conda/envs
               platform : osx-arm64
             user-agent : conda/23.7.4 requests/2.31.0 CPython/3.10.10 Darwin/22.6.0 OSX/13.6.2 solver/libmamba conda-libmamba-solver/23.3.0 libmambapy/1.5.1
                UID:GID : 17723:36677
             netrc file : None
           offline mode : False
@guyer guyer added the bug label Dec 10, 2023
@guyer
Copy link
Author

guyer commented Dec 10, 2023 •
edited
Loading

Environment created with previous release is OK

conda create --channel conda-forge --name scotch_lt_7 "scotch<7.0" petsc4py

Installed packages

# packages in environment at /Users/guyer/mambaforge-arm/envs/scotch_lt_7:
#
# Name                    Version                   Build  Channel
bzip2                     1.0.8                h93a5062_5    conda-forge
c-ares                    1.23.0               h93a5062_0    conda-forge
ca-certificates           2023.11.17           hf0a4a13_0    conda-forge
fftw                      3.3.10          mpi_openmpi_haef8dc3_8    conda-forge
gmp                       6.3.0                h965bd2d_0    conda-forge
hdf5                      1.14.3          mpi_openmpi_h20f603a_0    conda-forge
hypre                     2.28.0          mpi_openmpi_haba3941_0    conda-forge
icu                       73.2                 hc8870d7_0    conda-forge
krb5                      1.21.2               h92f50d5_0    conda-forge
libaec                    1.1.2                h13dd4ca_1    conda-forge
libblas                   3.9.0           20_osxarm64_openblas    conda-forge
libcblas                  3.9.0           20_osxarm64_openblas    conda-forge
libcurl                   8.5.0                h2d989ff_0    conda-forge
libcxx                    16.0.6               h4653b0c_0    conda-forge
libedit                   3.1.20191231         hc8eb9b7_2    conda-forge
libev                     4.33                 h93a5062_2    conda-forge
libexpat                  2.5.0                hb7217d7_1    conda-forge
libffi                    3.4.2                h3422bc3_5    conda-forge
libgfortran               5.0.0           13_2_0_hd922786_1    conda-forge
libgfortran5              13.2.0               hf226fd6_1    conda-forge
libhwloc                  2.9.3           default_h4394839_1009    conda-forge
libiconv                  1.17                 he4db4b2_0    conda-forge
liblapack                 3.9.0           20_osxarm64_openblas    conda-forge
libnghttp2                1.58.0               ha4dd798_1    conda-forge
libopenblas               0.3.25          openmp_h6c19121_0    conda-forge
libsqlite                 3.44.2               h091b4b1_0    conda-forge
libssh2                   1.11.0               h7a5bd25_0    conda-forge
libxml2                   2.11.6               h0d0cfa8_0    conda-forge
libzlib                   1.2.13               h53f4e23_5    conda-forge
llvm-openmp               17.0.6               hcd81f8e_0    conda-forge
metis                     5.1.0             h13dd4ca_1007    conda-forge
mpfr                      4.2.1                h9546428_0    conda-forge
mpi                       1.0                     openmpi    conda-forge
mumps-include             5.2.1               hce30654_11    conda-forge
mumps-mpi                 5.2.1               h51c74a9_11    conda-forge
ncurses                   6.4                  h463b476_2    conda-forge
numpy                     1.26.2          py312h5d55045_0    conda-forge
openmpi                   4.1.6              h526c993_101    conda-forge
openssl                   3.2.0                h0d3ecfb_1    conda-forge
parmetis                  4.0.3             h6eb5794_1005    conda-forge
petsc                     3.20.2          real_hcab63ac_100    conda-forge
petsc4py                  3.20.1          real_h17b5e51_100    conda-forge
pip                       23.3.1             pyhd8ed1ab_0    conda-forge
ptscotch                  6.0.9                h1aec651_2    conda-forge
python                    3.12.0          h47c9636_0_cpython    conda-forge
python_abi                3.12                    4_cp312    conda-forge
readline                  8.2                  h92ec313_1    conda-forge
scalapack                 2.2.0                h515df86_1    conda-forge
scotch                    6.0.9                h7537618_2    conda-forge
setuptools                68.2.2             pyhd8ed1ab_0    conda-forge
suitesparse               5.10.1               h7cd81ec_1    conda-forge
superlu                   5.2.2                hc615359_0    conda-forge
superlu_dist              7.2.0                hfdb8677_0    conda-forge
tbb                       2021.11.0            h6aa02a4_0    conda-forge
tk                        8.6.13               h5083fa2_1    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
wheel                     0.42.0             pyhd8ed1ab_0    conda-forge
xz                        5.2.6                h57fd34a_0    conda-forge
yaml                      0.2.5                h3422bc3_2    conda-forge
zlib                      1.2.13               h53f4e23_5    conda-forge
zstd                      1.5.5                h4f39d0f_0    conda-forge

Environment info

     active environment : scotch_lt_7
    active env location : /Users/guyer/mambaforge-arm/envs/scotch_lt_7
            shell level : 2
       user config file : /Users/guyer/.condarc
 populated config files : /Users/guyer/mambaforge-arm/.condarc
                          /Users/guyer/.condarc
          conda version : 23.7.4
    conda-build version : not installed
         python version : 3.10.10.final.0
       virtual packages : __archspec=1=arm64
                          __osx=13.6.2=0
                          __unix=0=0
       base environment : /Users/guyer/mambaforge-arm  (writable)
      conda av data dir : /Users/guyer/mambaforge-arm/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/osx-arm64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/osx-arm64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/osx-arm64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /Users/guyer/mambaforge-arm/pkgs
                          /Users/guyer/.conda/pkgs
       envs directories : /Users/guyer/mambaforge-arm/envs
                          /Users/guyer/.conda/envs
               platform : osx-arm64
             user-agent : conda/23.7.4 requests/2.31.0 CPython/3.10.10 Darwin/22.6.0 OSX/13.6.2 solver/libmamba conda-libmamba-solver/23.3.0 libmambapy/1.5.1
                UID:GID : 17723:36677
             netrc file : None
           offline mode : False

@dalcinl
Copy link
Contributor

dalcinl commented Dec 10, 2023

Let see if conda-forge/petsc4py-feedstock#84 solves the issue.

@dalcinl
Copy link
Contributor

dalcinl commented Dec 10, 2023

Indeed, there seem to be issues with macOS, I can see the Symbol not found: _SCOTCH_errorPrint in the build logs.
Unfortunately, I will not have time to figure out what's going on until end January. You will have to ask other petsc/petsc4py recipe maintainers for help.

@guyer
Copy link
Author

guyer commented Dec 10, 2023

For the time being, I'm fine with pinning the previous release. I just wanted to be sure you knew about it.

@minrk
Copy link
Member

minrk commented Dec 11, 2023

I believe this is an issue in scotch itself where libscotcherr wasn't linked from libscotch (possibly due to conda-forge -Wl,dead_strip_dylibs combined with something weird in how libscotch splits its functionality), but it should be fixed by the 7.0.4 migration, which should start today conda-forge/conda-forge-pinning-feedstock#4807

> otool -L ~/conda/envs/scotch702/lib/libscotch.dylib
/Users/minrk/conda/envs/scotch702/lib/libscotch.dylib:
	@rpath/libscotch.7.0.2.dylib (compatibility version 0.0.0, current version 7.0.2)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.0.0)
	@rpath/libz.1.dylib (compatibility version 1.0.0, current version 1.2.13)
	@rpath/libbz2.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/liblzma.5.dylib (compatibility version 8.0.0, current version 8.6.0)
> otool -L ~/conda/envs/scotch/lib/libscotch.dylib
/Users/minrk/conda/envs/scotch/lib/libscotch.dylib:
	@rpath/libscotch.7.0.4.dylib (compatibility version 0.0.0, current version 7.0.4)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.0.0)
	@rpath/libscotcherr.dylib (compatibility version 0.0.0, current version 0.0.0) <---- missing from 7.0.2
	@rpath/libz.1.dylib (compatibility version 1.0.0, current version 1.2.13)
	@rpath/libbz2.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/liblzma.5.dylib (compatibility version 8.0.0, current version 8.6.0)

@minrk minrk mentioned this issue Dec 11, 2023
Merged
@minrk
Copy link
Member

minrk commented Dec 11, 2023

7.0.4 builds should fix this when they arrive

@guyer
Copy link
Author

guyer commented Dec 12, 2023

7.0.4 build works for me, thanks

guyer added a commit to usnistgov/fipy that referenced this issue Dec 12, 2023
@guyer
Update locks to get new petsc
a480a84 
Skip over broken petsc on macOS
conda-forge/petsc-feedstock#180
guyer added a commit to usnistgov/fipy that referenced this issue Dec 13, 2023
@guyer
Environment pinning (#985)
2764950 
Pin environment definitions with [`conda-lock`](https://github.com/conda/conda-lock)

* Add conda environment definitions

* Add multi-platform lockfiles produced by `conda-lock lock`

* Add single-platform explicit lockfiles from `conda-lock render`

* Convert azure pipeline to use conda-lock lockfiles

* Pin PETSc on macOS

  Linkage between petsc 3.20.2_*_102 and scotch 7.0 is broken.
  conda-forge/petsc-feedstock#180

* Document different triggers

* Document use of lockfiles
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

scotch 7.0.4 minrk/petsc-feedstock
3 participants
@minrk @dalcinl @guyer