Skip to content

Commit

Permalink
Provide OPENBLAS_DEBUG option and some openblas build related cleanups.
Browse files Browse the repository at this point in the history
  • Loading branch information
@ViralBShah
ViralBShah committed Jul 1, 2013
1 parent 93e10d5 commit 1767dd9
Showing 1 changed file with 25 additions and 8 deletions.
33 changes: 25 additions & 8 deletions deps/Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -658,33 +658,45 @@ install-Rmath: $(RMATH_OBJ_TARGET)


## OpenBLAS ##
# LAPACK is built into OpenBLAS by default

OPENBLAS_OBJ_SOURCE = openblas-$(OPENBLAS_VER)/libopenblas.$(SHLIB_EXT)
OPENBLAS_OBJ_TARGET = $(BUILD)/lib/libopenblas.$(SHLIB_EXT)

OPENBLAS_BUILD_OPTS = CC="$(CC)" FC="$(FC)" RANLIB="$(RANLIB)" FFLAGS="$(FFLAGS) $(JFFLAGS)" USE_THREAD=$(OPENBLAS_USE_THREAD) TARGET=$(OPENBLAS_TARGET_ARCH)
ifeq ($(OPENBLAS_DYNAMIC_ARCH), 1)
OPENBLAS_BUILD_OPTS += DYNAMIC_ARCH=1
OPENBLAS_BUILD_OPTS = CC=clang FC="$(FC)" RANLIB="$(RANLIB)" FFLAGS="$(FFLAGS) $(JFFLAGS)" TARGET=$(OPENBLAS_TARGET_ARCH)

# Thread support
ifeq ($(OPENBLAS_USE_THREAD), 1)
OPENBLAS_BUILD_OPTS += USE_THREAD=1
endif

# Maximum number of threads for parallelism

This comment has been minimized.

Sign in to view

Copy link
@vtjnash

vtjnash Jul 12, 2013

Member

prior to this commit, we only allowed openblas to allocate enough threads to cover the number of processors on your system, unless OPENBLAS_DYNAMIC_ARCH=1 in which case we set it to the maximum for your platform. Is there a reason you changed this to always using the maximum?

This comment has been minimized.

Sign in to view

Copy link
@ViralBShah

ViralBShah Jul 13, 2013

Author Member

That explains why the crashes due to lots of threads were not visible otherwise and only for people who used precompiled binaries. I missed that point earlier, but I now think that having everyone use the maximum would have helped detect the issue earlier. I would prefer to leave it the way it is now. Any reason not to?

This comment has been minimized.

Sign in to view

Copy link
@vtjnash

vtjnash Jul 15, 2013

Member

It wastes memory and is a bit slower.

This comment has been minimized.

Sign in to view

Copy link
@JeffBezanson

JeffBezanson Jul 15, 2013

Member

Is this just a cap on the number of threads, or is it the number of threads it will actually use by default?

This comment has been minimized.

Sign in to view

Copy link
@ViralBShah

ViralBShah Jul 15, 2013

Author Member

It is a cap on the number of threads. How much memory is wasted? Since most users will likely use julia from a linux distribution or precompiled binaries, which will be compiled in such a way, I think it is reasonable for developers to use the same configuration. Otherwise, we risk avoiding bugs during development. For those who prefer to save memory on their computers, we can support changing of the defaults in Make.user.

This comment has been minimized.

Sign in to view

Copy link
@vtjnash

vtjnash Jul 15, 2013

Member

If my observations are correct, it will start that many threads, but the cap on the number that are in use is determined by set_blas_threads or CPU_CORES. Each thread has a stack of 16 MB, although only the first few pages are ever touched (since they aren't in use) so the actual waste is probably on the order of 2MB.

This comment has been minimized.

Sign in to view

Copy link
@vtjnash

vtjnash Jul 15, 2013

Member

Note that we have set NUM_THREADS=128 because we found that with higher numbers it appears that the OS would stop allowing us to allocate enough memory/threads and thus cause crashes (and limited more on 32-bit architectures and windows).
ifneq ($(ARCH),x86_64)
# assume we can't address much memory to spawn many threads
# Assume we can't address much memory to spawn many threads
OPENBLAS_BUILD_OPTS += NUM_THREADS=30
else
# assume we may want lots of threads on some future machine
# Assume we may want lots of threads on some future machine
ifeq ($(OS),WINNT) #windows seems unable to handle very many
OPENBLAS_BUILD_OPTS += NUM_THREADS=30
else #linux can handle up to 256 according to the FAQ
OPENBLAS_BUILD_OPTS += NUM_THREADS=256
endif
else # Linux can handle up to 256 according to the FAQ. Use 128 for now as that should suffice everywhere.
OPENBLAS_BUILD_OPTS += NUM_THREADS=128
endif
endif

# don't touch scheduler affinity since we manage this ourselves
OPENBLAS_BUILD_OPTS += NO_AFFINITY=1

# Build for all architectures - required for distribution
ifeq ($(OPENBLAS_DYNAMIC_ARCH), 1)
OPENBLAS_BUILD_OPTS += DYNAMIC_ARCH=1
endif

# 64-bit BLAS interface
ifeq ($(USE_BLAS64), 1)
OPENBLAS_BUILD_OPTS += INTERFACE64=1
endif

# Decide whether to build for 32-bit or 64-bit arch
ifneq ($(BUILD_OS),$(OS))
OPENBLAS_BUILD_OPTS += OSNAME=$(OS) CROSS=1 HOSTCC=$(HOSTCC)
endif
Expand All @@ -702,6 +714,11 @@ else
$(error "unknown arch for openblas cross-compile")
endif

# Debug OpenBLAS
ifeq ($(OPENBLAS_DEBUG), 1)
OPENBLAS_BUILD_OPTS += DEBUG=1
endif

openblas-$(OPENBLAS_VER).tar.gz:
$(WGET_DASH_O) $@ https://github.com/xianyi/OpenBLAS/tarball/$(OPENBLAS_VER)
openblas-$(OPENBLAS_VER)/config.status: openblas-$(OPENBLAS_VER).tar.gz
Expand Down

0 comments on commit 1767dd9

Please sign in to comment.