DIRAC 18.0 for LX

Webpage

http://www.diracprogram.org/doku.php

Version

18.0

Build Environment

• Intel Compiler 2019.1.144
  • ifort 19.0.1.144
  • icc 19.0.1.144
  • icpc 19.0.1.144
• Intel MKL 2019.0.1
• cmake 3.8.2

Files Required

• DIRAC-18.0-Source.tar.gz
• openmpi-3.1.0.tar.bz2
  • (special build is required to enable 8-bytes default integer size)

Build Procedure

#!/bin/sh

VERSION=18.0
INSTALL_PREFIX=/local/apl/lx/dirac180

# openmpi (8-byte integer)
OMPI_VERSION=3.1.0
OMPI_INSTALL_PREFIX=${INSTALL_PREFIX}/openmpi310_i8
OMPI_TARBALL=/home/users/${USER}/Software/OpenMPI/${OMPI_VERSION}/openmpi-${OMPI_VERSION}.tar.bz2
PBSROOT=/local/apl/lx/pbs14

# dirac
BASEDIR=/home/users/${USER}/Software/DIRAC/${VERSION}
TARBALL=${BASEDIR}/DIRAC-${VERSION}-Source.tar.gz

WORKDIR=/work/users/${USER}

PARALLEL=8
#-------------------------------------------------------------------------
umask 0022

export LC_ALL=C
export LANG=C
export OMP_NUM_THREADS=1

ulimit -s unlimited

module purge
module load intel/19.0.1
module load mkl/2019.0.1
module load cmake/3.8.2

# openmpi (8-byte integer default)
cd ${WORKDIR}
if [ -d openmpi-${OMPI_VERSION} ]; then
  mv openmpi-${OMPI_VERSION} openmpi_erase
  rm -rf openmpi_erase &
fi

tar jxf ${OMPI_TARBALL}
cd openmpi-${OMPI_VERSION}
mkdir rccs-i8 && cd rccs-i8
CC=icc CXX=icpc FC=ifort FCFLAGS=-i8  CFLAGS=-m64  CXXFLAGS=-m64 \
  ../configure --prefix=${OMPI_INSTALL_PREFIX} \
               --with-tm=${PBSROOT} \
               --enable-mpi-cxx \
               --with-psm2
make -j ${PARALLEL} && make install && make check

# dirac
cd ${WORKDIR}
if [ -d DIRAC-${VERSION}-Source ]; then
  mv DIRAC-${VERSION}-Source DIRAC_erase
  rm -rf DIRAC_erase &
fi

export PATH="${OMPI_INSTALL_PREFIX}/bin:$PATH"
export LIBRARY_PATH="${OMPI_INSTALL_PREFIX}/lib:$LIBRARY_PATH"
export LD_LIBRARY_PATH="${OMPI_INSTALL_PREFIX}/lib:$LD_LIBRARY_PATH"

export DIRAC_TMPDIR=${WORKDIR}

tar zxf ${TARBALL}
cd DIRAC-${VERSION}-Source
./setup --mpi \
        --fc=mpif90 \
        --cc=mpicc \
        --cxx=mpicxx \
        --mkl=parallel \
        --int64 \
        --extra-fc-flags="-xHost -I${OMPI_INSTALL_PREFIX}/lib" \
        --extra-cc-flags="-xHost" \
        --extra-cxx-flags="-xHost" \
        --prefix=${INSTALL_PREFIX} \
        build.rccs
cd build.rccs
make -j ${PARALLEL} && make install

# serial test
export DIRAC_MPI_COMMAND="mpirun -np 1"
make test

# parallel test
export DIRAC_MPI_COMMAND="mpirun -np ${PARALLEL}"
make test

exit 0

Important Notice

You must cite reference(s) listed in http://www.diracprogram.org/doku.php?id=citation when your result is published. Please check /local/apl/lx/dirac180/LICENSE file for more details.

Notes

• MPI is enabled, 8-bytes integer is used by default.
• OpenMP is enabled only for MKL functions. (MPI/OpenMP hybrid parallelization is possible.)
  • Hybrid parallel run information at http://www.diracprogram.org/doc/master/tutorials/hybrid_parallel_cc_run/hybrid_parallel_cc_run.html may be useful.
• OpenMPI 4.x can't be used with intact code. (Some of MPI functions used in DIRAC are obsolete and not available for OpenMPI 4.x or later.)
  • (update) might be possible if openmpi-4.x was built with --enable-mpi1-compatibility flag?

In this MPI enabled version, following tests failed regardless of the intel compiler version (checked for intel17, intel18, and intel19).

• general_property_krci (Failed)
• operators_mo_mtx_elements (Failed)

If multiple process are used (-np 2 or more), the following three tests also fail in addition to the above ones.

• polprp_ph (Failed)
• mp2_srdft_energies (Timeout)
• lucita_short (Failed)