Ability to use Intel provided libm

[ViralBShah]

It would be nice to have build options to use Intel's libm. If this is part of MKL, then we should try and use the Intel libm when building with USE_MKL.

[lindahua]

I remember it comes with MKL. I cannot confirm this though as I have the entire intel compiler suite in my system.

In all cases, we should allow the users to independently control this via another option, such as USE_INTEL_LIBM. If we confirm that it comes with MKL, we can set USE_INTEL_LIBM to be true by default when USE_MKL is set to true.

[lindahua]

We should also seriously consider ship Julia with Intel libraries. Their license allows you the redistribute anything in their redistributable list once you purchase the library (http://software.intel.com/en-us/articles/intel-math-kernel-library-licensing-faq#lic_purchase).

[blakejohnson]

libm does ship with MKL.

[ViralBShah]

We can ship Julia with Intel libraries on platforms we provide binaries - such as on Mac and Windows. However, there is no easy way to get these on linux (with OS packagers such as apt-get). I guess we can distribute MKL through BinDeps for such cases.

There is also the issue of requiring licenses for developers. It is not clear to me how that plays out in open source. Perhaps we keep openblas as default everywhere, except in mac and windows binaries. The downside there is that developers and users are using different libraries, and that is perhaps not a good idea.

[lindahua]

We may focus on getting the ability to use Intel LIBM first. Bundling Intel libraries can be considered later.

[nolta]

My read of intel's price sheet is that the licenses would cost $10K the first year and $4K every year thereafter, if we count everyone w/ commit access as a developer. And that's if we don't buy any OS X licenses -- those are 3x more expensive.

[blakejohnson]

Only the developer(s) responsible for building the distributions need a license, plus anyone that wants to test it that cannot obtain a license through other means.

On Saturday, July 20, 2013 at 10:53 AM, Mike Nolta wrote:

My read of intel's price sheet is that the licenses would cost $10K the first year and $4K every year thereafter, if we count everyone w/ commit access as a developer. And that's if we don't buy any OS X licenses -- those are 3x more expensive.

—
Reply to this email directly or view it on GitHub (#3772 (comment)).

[nolta]

"The number of Intel MKL copies that you need is determined by the number of developers who are writing code, compiling, and testing using Intel MKL API"

[blakejohnson]

Which by appropriate policy choice can be exactly one person. You have N developers using the netlib/openblas API and one developer using the MKL API for building/testing releases.

I had a conversation with Intel sales about this very issue a couple months ago.

On Saturday, July 20, 2013 at 11:04 AM, Mike Nolta wrote:

"The number of Intel MKL copies that you need is determined by the number of developers who are writing code, compiling, and testing using Intel MKL API"

—
Reply to this email directly or view it on GitHub (#3772 (comment)).

[nolta]

That's a pleasant surprise! Can you get it in writing?

[ViralBShah]

@lindahua Do you know which library in MKL to link to for libm? Would help save me some reading of docs. The libmkl_rt that we link to currently seems to have symbols for vectorized functions (vssin, etc.).

[blakejohnson]

@nolta I'd be happy to talk to Intel about our particular situation.

[lindahua]

Intel's math library is libimf.

[vharavy]

On Windows we Intel C++ Composer provides several libm implementations:

1. libm.lib - static library.
2. libmmt.lib - multi-threaded static library.
3. libmmd.lib and libmmd.dll - multi-threaded dynamic library. There are debug versions of these twop.
4. libmmds.lib - multi-threaded static library built with /MD

P.S. By the way the prices are not so high. Intel Composer XE 2013 costs $1,199 (for Windows) and includes C++ and Fortran support. If you do not need Fortran - Intel C++ Composer XE 2013 costs $699.

[lindahua]

Here is the relevant doc by intel:
http://software.intel.com/sites/products/documentation/doclib/iss/2013/compiler/cpp-lin/GUID-D7A8B891-FD72-440B-A00D-22F5AC0EF841.htm

In Linux:
libimf.a - static library
libimf.so - shared library

In Mac OS X:
libimf.a - static library
libimf.dylib - shared library

Function list: http://software.intel.com/sites/products/documentation/doclib/iss/2013/compiler/cpp-lin/GUID-837EC429-435C-4381-9393-BEED5C74A076.htm#GUID-837EC429-435C-4381-9393-BEED5C74A076

[ViralBShah]

@lindahua I went with the easier route for now - similar to what we do for BLAS and LAPACK. At least when you do USE_MKL, you get the Intel libm too.

[ViralBShah]

With libopenlibm:

julia> x = rand(10^8);
julia> @time log(x);
elapsed time: 3.761020889 seconds (800117392 bytes allocated)

With libimf

julia> x = rand(10^8);
julia> @time log(x);
elapsed time: 2.918278415 seconds (800000096 bytes allocated)

[nolta]

@ViralBShah, how much of the improvement is just due to openlibm being slow?

For example, on my macbook, your test runs in 2.4s using openlibm, but 1.9s using standard libm.

[ViralBShah]

Mac has a very good libm implementation, with implementations of many functions in assembly and using SSE instructions. Unfortunately, the apple open source license makes it difficult for us to incorporate in openlibm.

Openlibm is more generic, and may be comparable to glibc in performance.