Skip to content

Installing TAU

Jump to bottom
Kevin Huck edited this page Sep 15, 2019 · 9 revisions

Quick links:

A Note about TAU Dependencies

All TAU "dependencies" are optional - TAU can be configured and compiled with just ./configure && make -j install. However, that configuration will not be very useful - there will be no support for threads, MPI, etc. CUDA, PAPI and PDT support is described below, but all three are optional. If your code doesn't use CUDA, then you don't have to configure TAU with -cuda or load the module (where applicable). TAU is quite capable with the recommended set of configuration options: -bfd=download -dwarf=download -elf=download -unwind=download -otf=download -pthread -mpi -iowrapper.

Dependencies

The only build dependency that TAU requires to be built before TAU is PDT, if you want to auto-instrument code. TAU can instrument code without PDT, but it will use compiler-based instrumentation which has historically added large amounts of measurement overhead. PDT is a source-to-source code instrumentation utility that will add TAU instrumentation to source code. For more information on PDT, see https://tau.uoregon.edu/pdt. TAU can also use Dyninst, PEBIL or MAQAO for binary instrumentation (not described here). For more information on binary instrumentation, please contact tau-bugs@cs.uoregon.edu. Spack will automatically build PDT before building TAU. The following instructions are for manual installation.

To build PDT for code instrumentation, do the following (assumes GCC compilers, for other compilers see the output from ./configure -help):

wget http://tau.uoregon.edu/pdt.tgz
tar -xvzf pdt.tgz
cd pdtoolkit-3.25
./configure -GNU -prefix=/path/to/installation
make -j
make install

After installation, PDT can be added to the TAU configuration line with -pdt=/path/to/installation -pdt_c++=g++. Because the instrumentation tools are front-end tools, they can be compiled with GCC and do not have to be compiled with the same complier used to compile the measurement library. However, the compiler used to build PDT should be specified to the TAU configuration, unless the full stack is built with the same compilers.

Installing TAU with Spack

TAU configuration should auto-detect the architecture that it is building on, and with systems that require cross-compilation (Cray) it will build the measurement library for the compute nodes and the front-end tools for the login nodes. To configure/install TAU, do one of the following:

TAU installation

...with Spack

With a recent Spack (http://github.com:spack/spack) installation, do:

spack install tau@develop

By default, that will build TAU with binutils, libdwarf, libelf, libunwind (all used for sampling, call site resolution and compiler-based instrumentation for address resolution), otf2 (for tracing support), papi (hardware counter support), PDT (auto-instrumentation support), pthread (for threading support). That is, the TAU configuration in Spack will look something like: tau@develop%gcc@8.1.0~bgq+binutils~comm~craycnl~cuda~gasnet+libdwarf+libelf+libunwind~likwid~mpi~ompt~opari~openmp+otf2+papi+pdt~phase~ppc64le+pthreads~python~scorep~shmem~x86_64. If your only interest is in using TAU to measure ADIOS2, you can save time/space by installing TAU without PDT:

spack install tau@develop~pdt

Note: As of September 5, 2019, the TAU spack package was missing support for the POSIX I/O wrapper. That's been requested as a pull request, but if not yet merged and you want POSIX I/O wrapper support, use this installation package: https://github.com/khuck/spack/blob/develop/var/spack/repos/builtin/packages/tau/package.py

Here is a summary of the TAU info message in Spack:

$ spack info tau
Package:   tau

Description:
    A portable profiling and tracing toolkit for performance analysis of
    parallel programs written in Fortran, C, C++, UPC, Java, Python.

Homepage: http://www.cs.uoregon.edu/research/tau

Tags: 
    None

Preferred version:  
    2.28.2      https://www.cs.uoregon.edu/research/tau/tau_releases/tau-2.28.2.tar.gz

Safe versions:  
    develop     [git] https://github.com/UO-OACISS/tau2 on branch master
    2.28.2      https://www.cs.uoregon.edu/research/tau/tau_releases/tau-2.28.2.tar.gz
    2.28.1      https://www.cs.uoregon.edu/research/tau/tau_releases/tau-2.28.1.tar.gz
    2.28        https://www.cs.uoregon.edu/research/tau/tau_releases/tau-2.28.tar.gz
    2.27.2p1    https://www.cs.uoregon.edu/research/tau/tau_releases/tau-2.27.2p1.tar.gz
    2.27.2      https://www.cs.uoregon.edu/research/tau/tau_releases/tau-2.27.2.tar.gz
    2.27.1      https://www.cs.uoregon.edu/research/tau/tau_releases/tau-2.27.1.tar.gz
    2.27        https://www.cs.uoregon.edu/research/tau/tau_releases/tau-2.27.tar.gz
    2.26.3      https://www.cs.uoregon.edu/research/tau/tau_releases/tau-2.26.3.tar.gz
    2.26.2      https://www.cs.uoregon.edu/research/tau/tau_releases/tau-2.26.2.tar.gz
    2.26.1      https://www.cs.uoregon.edu/research/tau/tau_releases/tau-2.26.1.tar.gz
    2.26        https://www.cs.uoregon.edu/research/tau/tau_releases/tau-2.26.tar.gz
    2.25        https://www.cs.uoregon.edu/research/tau/tau_releases/tau-2.25.tar.gz
    2.24.1      https://www.cs.uoregon.edu/research/tau/tau_releases/tau-2.24.1.tar.gz
    2.24        https://www.cs.uoregon.edu/research/tau/tau_releases/tau-2.24.tar.gz
    2.23.1      https://www.cs.uoregon.edu/research/tau/tau_releases/tau-2.23.1.tar.gz

Variants:
    Name [Default]    Allowed values    Description


    bgq [off]         True, False       Build for IBM BlueGene/Q
                                        compute nodes
    binutils [on]     True, False       Activates support of BFD GNU
                                        Binutils
    comm [off]        True, False       Generate profiles with MPI
                                        communicator info
    craycnl [off]     True, False       Build for Cray compute nodes
    cuda [off]        True, False       Activates CUDA support
    fortran [on]      True, False       Activates Fortran support
    gasnet [off]      True, False       Activates GASNET support
    io [on]           True, False       Activates POSIX I/O support
    libdwarf [on]     True, False       Activates support of libdwarf
    libelf [on]       True, False       Activates support of libelf
    libunwind [on]    True, False       Activates support of libunwind
    likwid [off]      True, False       Activates LIKWID support
    mpi [off]         True, False       Specify use of TAU MPI wrapper
                                        library
    ompt [off]        True, False       Activates OMPT instrumentation
    opari [off]       True, False       Activates Opari2
                                        instrumentation
    openmp [off]      True, False       Use OpenMP threads
    otf2 [on]         True, False       Activates support of Open
                                        Trace Format (OTF)
    papi [on]         True, False       Activates Performance API
    pdt [on]          True, False       Use PDT for source code
                                        instrumentation
    phase [off]       True, False       Generate phase based profiles
    ppc64le [off]     True, False       Build for IBM Power LE nodes
    pthreads [on]     True, False       Use POSIX threads
    python [off]      True, False       Activates Python support
    scorep [off]      True, False       Activates SCOREP support
    shmem [off]       True, False       Activates SHMEM support
    x86_64 [off]      True, False       Force build for x86 Linux
                                        instead of auto-detect

Installation Phases:
    install

Build Dependencies:
    binutils  cuda  gasnet  libdwarf  libelf  libunwind  likwid  mpi  otf2  papi  pdt  python  scorep

Link Dependencies:
    binutils  cuda  gasnet  libdwarf  libelf  libunwind  likwid  mpi  otf2  papi  pdt  python  scorep

Run Dependencies:
    None

Virtual Packages: 
    None

Installing TAU on Summit (ORNL)

TAU will download and build most dependencies, with the exception of libraries such as PAPI and CUDA - usually pre-installed on systems. TAU assumes the GCC compilers by default. For ABI compatibility, TAU requires you build with the same compiler and MPI configuration used with ADIOS2 and the application(s). By default TAU will install in-place, in tau2/$arch.

An important note: TAU is (currently) incompatible with Darshan. So before building TAU or using it on the compute nodes, the Darshan runtime module must be unloaded.

module unload darshan-runtime
module load gcc/6.4.0 papi/5.6.0 cuda/9.2
git clone https://github.com/UO-OACISS/tau2.git
cd tau2
./configure -bfd=download -dwarf=download -elf=download -unwind=download -otf=download -papi=${OLCF_PAPI_ROOT} -pthread -mpi -iowrapper -cuda=${OLCF_CUDA_ROOT} -prefix=/path/to/installation
make -j install

When using different compilers (i.e. PGI), you specify them on the configuration line like so (for historical reasons, the fortran compiler is specified by vendor, not name...in most cases):

module unload darshan-runtime
module load papi/5.6.0 cuda/9.2
./configure -cc=pgcc -c++=pgc++ -fortran=pgi -bfd=download -dwarf=download -elf=download -unwind=download -otf=download -papi=${OLCF_PAPI_ROOT} -pthread -mpi -iowrapper -cuda=${OLCF_CUDA_ROOT} -prefix=/path/to/installation
make -j install

OpenACC Support

TAU will automatically detect OpenACC support with the compiler/platform. See the tau2/examples/openacc directory for details and an example.

OpenMP Support

TAU has two options for support with OpenMP - OPARI instrumentation or OMPT. For olderIntel, LLVM and GCC compilers, TAU can download and build the modern LLVM OpenMP runtime and use that for OMPT measurement. Newer compilers (LLVM 8.0+, Intel 19+) have OMPT support built-in to the runtime. TAU will detect whether the Compiler has OMPT support, and use it if available. If not, it will build and use the LLVM library. When using other compilers (i.e. PGI, XL, etc.) the only other option is OPARI. For OPARI support, see tau2/examples/opari for more information.

To configure TAU with OpenMP and OMPT support, replace -pthread with -openmp -ompt:

module unload darshan-runtime
module load gcc/6.4.0 papi/5.6.0 cuda/9.2
./configure -bfd=download -dwarf=download -elf=download -unwind=download -otf=download -papi=${OLCF_PAPI_ROOT} -openmp -ompt -mpi -iowrapper -cuda=${OLCF_CUDA_ROOT} -prefix=/path/to/installation
make -j install

Installing TAU on Cori (LBL)

Get and build PDT

The default compilers on Cori are Intel 19 (as of Sept. 13, 2019). However, we'll build PDT with the default GNU compilers in /usr/bin, which happen to be GCC 7.4 (again, as of Sept. 13, 2019).

cd $HOME/src
wget http://tau.uoregon.edu/pdt.tgz
tar -xzf pdt.tgz
cd pdtoolkit-3.25.1
./configure -GNU -prefix=/path/to/install/pdt
make
make install

Get and build TAU

TAU will be compiled with the default Intel compilers, using the Cray compiler wrappers. However, the login-node tools will be compiled with GCC (because we built PDT with GCC).

module unload darshan
module load papi/5.6.0.6
cd $HOME/src
git clone https://github.com/UO-OACISS/tau2.git
cd tau2
./configure -bfd=download -dwarf=download -elf=download -unwind=download -otf=download -papi=/opt/cray/pe/papi/5.6.0.6 -pthread -mpi -iowrapper -cc=icc -c++=icpc -fortran=intel -prefix=/path/to/install/tau -pdt=/path/to/install/pdt/GNU -pdt_c++=g++
make -j install

Installing TAU on Theta (ANL)

TBD

Installing TAU on x86_64 Linux systems

TBD

Still have questions? Check out the official documentation or contact tau-bugs@cs.uoregon.edu for help.
Clone this wiki locally