OpenBLAS crash when using OPENBLAS_DYNAMIC_ARCH=1

[ViralBShah]

When I build julia mac binaries, I set OPENBLAS_DYNAMIC_ARCH=1. I find that running peakflops() leads to this error and subsequent calls lead to a segfault.

julia> peakflops()
ERROR: stack overflow
 in gemm! at linalg/blas.jl:347
 in gemm_wrapper at linalg/matmul.jl:302
 in gemm_wrapper at linalg/matmul.jl:283
 in peakflops at util.jl:47
 in peakflops at no file

This does not happen if one disables multi-threading by using only one OpenBLAS thread. This also does not happen when OPENBLAS_DYNAMIC_ARCH is not being used.

I have filed an upstream issue (OpenMathLib/OpenBLAS#221), but the issue is a bit vague at this point.

[ViralBShah]

Current mac binaries set OPENBLAS_NUM_THREADS=1 before startup as a temporary workaround. See 0e411ad

[ViralBShah]

The openblas tests run fine for me but the basic matrix multiply with multi-threading turned on crashes in julia.

@JeffBezanson Is there a way to make ccall print some debugging information.

[ViralBShah]

Also Cc: @xianyi

[staticfloat]

A little late to the party, but yes, I can reproduce this using the latest on Homebrew. I'm going through the instructions provided by xianyi in the linked thread, specifically, trying the develop branch and with DEBUG=1 set.

[ViralBShah]

@staticfloat Were you able to debug this further?

Do you think we should build two mac nightlies for now - one with openblas and one with Apple BLAS? Apple BLAS does work multi-threaded, but I have to try out how well it does with small problems and such.

[staticfloat]

No, I wasn't. I got bogged down because of the inability of gcc's assembler to use AVX codes, (finally figured out that you need to use clang, but then I got sidetracked by other projects).

I can definitely build an Accelerate OSX binary as well..... but when you do this testing, could you post the .jl files you used to do the testing so I can put them into codespeed? That's exactly the kind of thing that I think it'd be fun to keep track of.

[ViralBShah]

It may be time to write a BLAS performance test (#3566).

[ViralBShah]

This is fixed by using 128 instead of 256 max threads in 1767dd9

[ViralBShah]

Upstream issue remains filed - but julia related issue is addressed for now.

[staticfloat]

Thanks for all this work, @ViralBShah!