Skip to content

REX with UPIR development

Jump to bottom
Yonghong Yan edited this page Feb 1, 2023 · 19 revisions

1. Prerequisite

1.1 OS:

Ubuntu 22.04/20.04 LTS with GCC 9.x is tested. Ubuntu 18.04 or older is not supported. GCC 10.x or later is not supported.

1.2 Software:

sudo apt update && \
sudo apt install -y openjdk-8-jdk && \
sudo apt install -y \
        antlr4 \
        autoconf \
        automake \
        autotools-dev \
        bc \
        binutils \
        bison \
        build-essential \
        cmake \
        cpufrequtils \
        curl \
        device-tree-compiler \
        dkms \
        doxygen \
        flex \
        gawk \
        gdb \
        gfortran \
        ghostscript \
        git \
        gperf \
        graphviz \
        libantlr4-runtime-dev \
        libboost-all-dev \
        libgmp-dev \
        libhpdf-dev \
        libmpc-dev \
        libmpfr-dev \
        libomp-dev \
        libtool \
        libxml2-dev \
        patchutils \
        perl-doc \
        python3-dev \
        sqlite \
        texinfo \
        unzip \
        vim \
        wget \
        zip \
        zlib1g \
        zlib1g-dev

sudo apt install gcc-multilib libelf-dev libffi-dev ninja-build

1.3 Environment

The compilation requires that CC and CXX point to the same compiler version. For example, it won't work if CC=gcc-5 and CXX=g++-7. On each of our systems, we used the shipped compiler.

export CC=/usr/bin/gcc
export CXX=/usr/bin/g++

However, if you wish to use another version, or find it necessary to do so, you can repeat the steps above:

export CC=/usr/bin/gcc-7
export CXX=/usr/bin/g++-7

So far OpenJDK 8 is tested. Even we remove the limit to JDK, other versions may not work as expected.

export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
export LD_LIBRARY_PATH=$JAVA_HOME/jre/lib/amd64/server:$LD_LIBRARY_PATH

Assume we prepare a folder $HOME/Projects/rexdev for the compilation and installation. You could use other proper folders as well.

export REX_ROOT=$HOME/Projects/rexdev

Enabling stacktrace for ASSERT_* and ROSE_ASSERT that are failed and for ROSE_ABORT.

Printing stacktrace at known failed location is convenient for debugging purpose since we do not need to use a debugger to see the stacktrace. We implemented this feature using boost stacktrace in mlog.C file. But it must be enabled in the configure/building of the whole source tree. To enable it, set CFLAGS/CXXFLAGS/LDFLAGS for configure or export them, e.g.

       export CXXFLAGS="-g -DBOOST_STACKTRACE_LINK -DREX_STACKTRACE_ABORT_BOOST"
       export CFLAGS="-g -DBOOST_STACKTRACE_LINK -DREX_STACKTRACE_ABORT_BOOST"
       export LDFLAGS="-lboost_stacktrace_backtrace"

Those compiler and linker lags can be passed to --with-DEBUG, -with-C_DEBUG, or -with-CXX_DEBUG of the configure, but setting directly via CFLAGS/CXXFLAGS/LDFLAGS seems to be easier to use. This is only tested on Ubuntu 22.04 with gcc-9/g++-9/gfortran-9 and system-installed boost

2. Build

2.1 Using Autotools (Recommended)

cd $REX_ROOT
git clone -b dev git@github.com:passlab/rexompiler.git rex_src
mkdir rex_build
cd rex_src
git submodule update --init
./build
cd ../rex_build
CC=gcc-9 CXX=g++-9 FC=gfortran-9 ../rex_src/configure --prefix=$REX_ROOT/rex_install --with-boost=/usr --with-boost-libdir=/usr/lib/x86_64-linux-gnu --enable-languages=c,c++,fortran --disable-tests-directory --disable-tutorial-directory --with-gomp_omp_runtime_library=/usr/lib/gcc/x86_64-linux-gnu/9
make core -j6
make install-core

2.2 Using CMake

CMake support in the REX compiler still has some issues. For example, CUDA cannot be enabled correctly. Tests for interfaces are not implemented.

cd $REX_ROOT
git clone -b dev git@github.com:passlab/rexompiler.git rex_src
mkdir rex_build
cd rex_src
git submodule update --init
cd ../rex_build
CC=gcc-9 CXX=g++-9 FC=gfortran-9 cmake -Denable-c=ON -Denable-fortran=ON -DCMAKE_INSTALL_PREFIX=../rex_install ../rex_src
cmake --build . -j6
make install

3. Usage

Now the compilation is completed and the REX Compiler is installed in $REX_ROOT/rex_install.

export LD_LIBRARY_PATH=$REX_ROOT/rex_install/lib:$LD_LIBRARY_PATH
export PATH=$REX_ROOT/rex_install/bin:$PATH

To generate the lowered source code of an OpenMP/OpenACC program foo.c:

rose-compiler -rose:openmp:lowering -rose:skipfinalCompileStep foo.c

Then we can use any C/C++ compiler to build the new source code and link it with LLVM OpenMP runtime.

4. Tests

There are at least three locations we need to test, which are /tests/nonsmoke/functional/CompileTests/OpenMP_tests, /tests/nonsmoke/functional/roseTests/astInterfaceTests, and /tests/nonsmoke/functional/roseTests/ompLoweringTests. However, at this moment we only enable the tests in the first two locations because the last one hasn't been modified for REXCompiler yet.

The following tests should be passed locally before committing.

cd $REX_ROOT/rex_build/tests/nonsmoke/functional/CompileTests/OpenMP_tests
make check
cd $REX_ROOT/rex_build/tests/nonsmoke/functional/roseTests/astInterfaceTests
make check

5. Miscellaneous

Please notice that GitHub supports two ways to access the code using https/password and git/SSH key. In this tutorial, the git/SSH key is used. To use the other method, all the git links should be replaced with the https link. Check here for more details.

6. Build and test with coverage analysis

ROSE provides building with coverage analysis using lcov. To build with coverage, add --enable-lcov to the configure script, which add -fprofile-arcs -ftest-coverage and -lgcov to CFLAGS/CXXFLAGS/LDFLAGS for gcc/g++. Then perform normal make, and test (core, OpenMP_tests and astInterfaceTest). Testing execution will cause coverage data to be produced in the .libs folder of source code folders. Then in the toplevel build folder, launch make lcov; and make lcov_genhtml which will produce the coverage summary HTML file in lcov_output folder.

7. Generate Doxygen HTML docs

The execution of make html command will produce the Doxygen HTML file in the docs/Rose/ROSE_WebPages/ROSE_HTML_Reference folder.

Clone this wiki locally