Build with MKL problem #2

[esd100]

After a successful build of Julia with gfortran 4.8.2 on OS X Mavericks (10.9) with Haswell processor using OpenBLAS 0.3.9rc2 (see #6272), I decided to delete everything and try to see if I could build Julia using MKL. After I deleted everything and installed MKL version (composer_xe_2013_sp1.2.139), I downloaded Julia source again, modified make.user and tried to build it.

The build process made it very far, as you can see from the length of the gist, (https://gist.github.com/esd100/9905847) but failed with the error below. :-(

libtool: link: ar cru .libs/libarpacksrc.a .libs/sgetv0.o .libs/slaqrb.o .libs/sstqrb.o .libs/ssortc.o .libs/ssortr.o .libs/sstatn.o .libs/sstats.o .libs/snaitr.o .libs/snapps.o .libs/snaup2.o .libs/snaupd.o .libs/snconv.o .libs/sneigh.o .libs/sngets.o .libs/ssaitr.o .libs/ssapps.o .libs/ssaup2.o .libs/ssaupd.o .libs/ssconv.o .libs/sseigt.o .libs/ssgets.o .libs/sneupd.o .libs/sseupd.o .libs/ssesrt.o .libs/dgetv0.o .libs/dlaqrb.o .libs/dstqrb.o .libs/dsortc.o .libs/dsortr.o .libs/dstatn.o .libs/dstats.o .libs/dnaitr.o .libs/dnapps.o .libs/dnaup2.o .libs/dnaupd.o .libs/dnconv.o .libs/dneigh.o .libs/dngets.o .libs/dsaitr.o .libs/dsapps.o .libs/dsaup2.o .libs/dsaupd.o .libs/dsconv.o .libs/dseigt.o .libs/dsgets.o .libs/dneupd.o .libs/dseupd.o .libs/dsesrt.o .libs/cnaitr.o .libs/cnapps.o .libs/cnaup2.o .libs/cnaupd.o .libs/cneigh.o .libs/cneupd.o .libs/cngets.o .libs/cgetv0.o .libs/csortc.o .libs/cstatn.o .libs/znaitr.o .libs/znapps.o .libs/znaup2.o .libs/znaupd.o .libs/zneigh.o .libs/zneupd.o .libs/zngets.o .libs/zgetv0.o .libs/zsortc.o .libs/zstatn.o
libtool: link: ranlib .libs/libarpacksrc.a
libtool: link: ( cd ".libs" && rm -f "libarpacksrc.la" && ln -s "../libarpacksrc.la" "libarpacksrc.la" )
Making all in .
libtool: link: gfortran -mmacosx-version-min=10.7 -dynamiclib -o .libs/libarpack.2.dylib -Wl,-force_load,./SRC/.libs/libarpacksrc.a -Wl,-force_load,./UTIL/.libs/libarpackutil.a -framework vecLib -mmacosx-version-min=10.7 -O2 -Wl,-rpath -Wl,/Applications/julia/usr/lib -framework vecLib -install_name /Applications/julia/usr/lib/libarpack.2.dylib -compatibility_version 3 -current_version 3.0 -Wl,-single_module
gfortran: fatal error: no input files; unwilling to write output files
compilation terminated.
make[4]: *** [libarpack.la] Error 1
make[3]: *** [all-recursive] Error 1
make[2]: *** [arpack-ng-3.1.3/.libs/libarpack.dylib] Error 2
make[1]: *** [julia-release] Error 2
make: *** [release] Error 2

[catawbasam]

Have you seen: http://software.intel.com/en-us/articles/julia-with-intel-mkl-for-improved-performance?

Even if the build tips there don't help, the poster is inviting feedback.

[ivarne]

That guide starts very badly by suggesting to download the zip file from github. That zip file does not work, because we use submodules for some of our dependencies.

We also have a script to extract information about the repo history for the versioninfo() function, that should fail silently.

[ivarne]

It also suggests editing Make.inc instead of overriding the USE_MKL in a Make.user file.

[esd100]

I did post on the website. While the guide itself is not great, I think the fact that Julia is on Intel's radar is a great thing for Julia. Only good things can come from fostering an improved relationship with Intel. Especially since the last I read, Xianyi is not even conveying any certainty in whether he will continue to develop OpenBLAS (see OpenMathLib/OpenBLAS#351)

[esd100]

Unfortunately, no updates yet...

[esd100]

Still ... unfortunately, no updates ...

[esd100]

I guess I should just give up on this issue

[andreasnoack]

I don't have MKL for Mac, but last week I built with MKL on Ubuntu without problems by defining Make.user with

USE_MKL=1
MKLROOT=/opt/intel/mkl

so could you try a fresh clone of Julia with a Make.user like that?

[tkelman]

You're going to have to find someone who has built Julia successfully with MKL on Mac. I think @ViralBShah is one such person. If he can recall around when it last worked, you can try checking out an old version of Julia and see if it works there, then git bisect to find when it stopped working.

Also, did you try posting in Intel's support forum, in addition to just commenting on a few-months-old article that they may not be checking?

The step that's failing is using the gfortran compiler as a linker driver to make a shared arpack library from the static .a arpack libraries, and linking Blas. However the -framework vecLib seems to indicate it's trying to link the Apple Accelerate Blas, instead of MKL.

[esd100]

Thanks for the tips. I'll try posting a on intel support forum and checking a new build from scratch. I think gcc should be releasing a stable 4.9 version around easter. I'll have some results by then I hope. I'm not sure why the build would be linking the Accelerate framework instead of MKL.

[tkelman]

Judging by the line checking for sgemm_ in -L/lib/intel64 -lmkl_rt... no, you apparently did not provide MKLROOT in Make.user (or it was not set by running /path/to/mkl/bin/mklvars.sh intel64 ilp64).

[ViralBShah]

I do not have MKL on mac either, and have never tried it.

[tkelman]

Ah, sorry then, nevermind. It looked like you had worked on some or all of the MKL portions of the Makefiles.

[ViralBShah]

Yes I have - but just not with MKL on mac. :-)

[esd100]

I'll check and see if that was the issue.

[esd100]

@tkelman Thanks Tony. You successfully diagnosed the problem. The MKLROOT environment variables were not set correctly in my .bash_profile.

See attached gist for further details. https://gist.github.com/esd100/10906796

[pao]

@esd100 Are there remaining unresolved issues or lack of documentation that need to be addressed here, or are we OK to close this?

[esd100]

Thanks for checking.

May I suggest the following update to documentation to using MKL on Mac OS X Mavericks.

1. open terminal application
2. open -a TextEdit .bash_profile (Note: this file may not already exist, but will be created)
3. add the following, then save
   MKLPATH=/opt/intel/mkl
   source ${MKLPATH}/bin/mklvars.sh intel64 ilp64
   export MKL_INTERFACE_LAYER=ILP64

[esd100]

Other than that, this issue is okay to close.

[pao]

@esd100 Would you be able to make a pull request adding that to the appropriate part of the README? Go here and click the "Edit" button in the header above the rendered file, make your changes, then follow the instructions to make a pull request. Refer to this issue number as #6367 in your request so it will be linked to this issue. Thanks!

[ivarne]

So essentially we should've somehowe made sure that the instructions in https://github.com/JuliaLang/julia#intel-math-kernel-libraries made sure that the user understood that he would have to run them every time before building with MKL.

[esd100]

Thanks for the instructions. I will try to make the changes and do the pull request thing. I have never done it before, so I will have to figure it out.

[esd100]

@ivarne I think maybe that it's just a little bit difficult to understand to the non-technically savvy person. i.e., people that haven't used configure, make, install commands very often or even people that haven't used a terminal very often..

[pao]

I have never done it before, so I will have to figure it out.

Thanks! Suggesting this is the sneaky way we try to get new contributors to Julia.

[ViralBShah]

Doesn't MKL provide a script to do this on mac like linux?