Segfault with Python #97
Comments
|
I was able to get a stacktrace: Reproducer: install pyscf in python and do I've verified this under julia 1.7 (linux), with both my system-provided python and julia's one |
|
@staticfloat I thought these issues would be behind us with LBT. |
|
I encounter something similar using This works: julia> using PyPlot
julia> using MKL
julia> plot(3:4)
1-element Vector{PyCall.PyObject}:
PyObject <matplotlib.lines.Line2D object at 0x7f71beacf430>This doesn't: julia> using MKL
julia> using PyPlot
julia> plot(3:4)
signal (11): Segmentation fault
in expression starting at none:0This is on julia> versioninfo()
Julia Version 1.7.1
Commit ac5cc99908 (2021-12-22 19:35 UTC)
Platform Info:
OS: Linux (x86_64-pc-linux-gnu)
CPU: Intel(R) Core(TM) i3-5005U CPU @ 2.00GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-12.0.1 (ORCJIT, broadwell)
Environment:
JULIA_EDITOR = vimand |
|
This looks possibly different, my segfault occurs no matter what order the packages are loaded |
|
@staticfloat @giordano Isn't this the usual problem of ILP64 MKL from Julia and LP64 from Python clashing? I believe the next MKL release that allows the |
|
We have worked with Intel MKL team and the fix has landed. This should hopefully give us what we need to fix all these issues. |
|
I've been trying to run |
|
I can confirm that using MKL_jll (and not using MKL) appears to fix both the MKL+PyPlot as well as the original bug report in this issue. |
|
Progress: JuliaLang/julia#43877 Next step would be to update MKL.jl as discussed in JuliaLinearAlgebra/libblastrampoline#54 (comment) |
|
The PyPlot example no longer crashes, irrespective of whether I now load MKL.jl before or after. If #104 is good and merges, this will conclude an extremely long project of working with Intel to bring 64_ suffixes to MKL for ILP64, and then rolling it all out in the Julia ecosystem all the way, enabled by all the work on LBT that @staticfloat and @giordano have done. |
|
This also paves the way for us to be able to link Ygg binaries to LBT - both ILP64 and LP64 BLAS - and have the ability to use MKL across the ecosystem. |
|
I seem to still encounter the (@v1.8) pkg> st -m MKL
Status `~/.julia/environments/v1.8/Manifest.toml`
[33e6dc65] MKL v0.4.4 `https://github.com/JuliaLinearAlgebra/MKL.jl#master`
(@v1.8) pkg> st -m PyPlot
Status `~/.julia/environments/v1.8/Manifest.toml`
[d330b81b] PyPlot v2.10.0
julia> using MKL; using PyPlot; plot(3:4);
signal (11): Segmentation fault
in expression starting at none:0
[1] 29598 segmentation fault (core dumped) julia-latest |
|
I think @ViralBShah was refering to ongoing work in PRs, not yet landed |
|
That's right, #104 fixes it. Would be great if you can try it out. There was one small issue we found that needs a workaround (JuliaLinearAlgebra/libblastrampoline#56), and after that we can merge it all. Quite hopeful that it will happen by 1.8. |
|
Thanks, I can confirm that the pyplot issue is fixed using that branch. Amazing work! Hoping that this makes it to 1.8 |
|
We expect it will make it to 1.8. |
|
FWIW, I still get segfaults with julia master and MKL.jl 0.5. Reproducer is the same as the second post above. |
|
We haven't been able to merge my PR yet, because Intel gave us ILP64 mangling for fortran names but not CBLAS names, and we need CBLAS for dot. @staticfloat handled this in LBT 5 - but #104 is failing with LBT 5. I hope it will get fixed by the time 1.8 releases. Also, Intel is working on adding CBLAS name mangling for the next release. |
|
@ViralBShah sorry I didn't test at the time of your PR, but the MWE at #97 (comment) is still failing with julia 1.8.2 and MKL 0.5.0 |
|
Could you test against master (MKL 0.6.0)? |
|
Oh sorry I missed that. Awesome, it works. Can you guys tag a new release? |
|
Bump? |
|
What needs to be done here? Didn't 0.6 work? |
|
I think the suggestion is that a release be tagged |
|
I thought I tagged v0.6, but Tagbot seems to have errored out and hence no tags or release notes in this repo. See the recent commits. |
I've run into similar issues in the past, feel free to close if that's known. When PyCalling into a package (pyscf), it segfaults (with nothing more informative, presumably when pyscf attempts to do linear algebra). I can provide more information if needed, but really it seems that it's just
using MKL; using PyCall; do_something_with_linalg_from_python().The text was updated successfully, but these errors were encountered: