GENESIS 2.0.3 with GPU support

Webpage

https://www.r-ccs.riken.jp/labs/cbrt/

Version

2.0.3

Build Environment

  • gcc 11.2.1 (gcc-toolset/11)
  • MKL 2022.2.1
  • HPC-X 2.11 (Open MPI 4.1.4)
    • (HPC-X 2.13.1 was used in the actual build. However, that version caused a problem when large number of MPI processes were employed. Switch runtime library to HPC-X 2.11 solved the problem.)
    • (In the following, assuming HPC-X 2.11 is used instead of HPC-X 2.13.1.)
  • CUDA 11.6

Files Required

  • genesis-2.0.3.tar.bz2
  • tests-2.0.3.tar.bz2

Build Procedure

#!/bin/sh

VERSION=2.0.3
BASEDIR=/home/users/${USER}/Software/GENESIS/${VERSION}
SRC_TARBALL=${BASEDIR}/genesis-${VERSION}.tar.bz2
TESTS_TARBALL=${BASEDIR}/tests-${VERSION}.tar.bz2

INSTALLDIR=/apl/genesis/2.0.3-CUDA

WORKDIR=/gwork/users/${USER}
BUILDDIR=${WORKDIR}/genesis-${VERSION}
TESTSDIR=${WORKDIR}/tests-${VERSION}

PARALLEL_TESTS=8

# ----------------------------------------------------------------------------
umask 0022

module -s purge
module -s load gcc-toolset/11
module -s load mkl/2022.2.1
module -s load openmpi/4.1.5-hpcx/gcc11
module -s load cuda/11.6
export CUDA_VISIBLE_DEVICES=0

export LANG=C
export LC_ALL=C
export OMP_NUM_THREADS=1
#ulimit -s unlimited

cd ${WORKDIR}
if [ -d genesis-${VERSION} ]; then
  mv genesis-${VERSION} genesis-erase
  rm -rf genesis-erase &
fi

if [ -d tests-${VERSION} ]; then
  mv tests-${VERSION} tests-erase
  rm -rf tests-erase &
fi

tar jxf ${SRC_TARBALL}
tar jxf ${TESTS_TARBALL}

cd ${BUILDDIR}

# sed -i -e 6942i"GENCODEFLAG+=' --generate-code=arch=compute_80,code=\"sm_80,compute_80\"'" configure

FC=mpif90 CC=mpicc \
LAPACK_LIBS=" -L${MKLROOT}/lib/intel64 -Wl,--no-as-needed -lmkl_gf_lp64 -lmkl_gnu_thread -lmkl_core -lgomp -lpthread -lm -ldl" \
  ./configure --prefix=${INSTALLDIR} \
              --enable-gpu \
              --enable-single \
              --with-cuda=/apl/cuda/11.6

make && make install

SPDYN=${INSTALLDIR}/bin/spdyn

cd ${TESTSDIR}/regression_test

for f in test.py test_remd.py test_rpath.py test_gamd.py; do
  sed -i -e "s/env python/env python3/" $f
done
sed -i -e "s/env python/env python2/" test_nonstrict.py

# spdyn tests
./test.py           "mpirun -np ${PARALLEL_TESTS} $SPDYN"
./test_remd.py      "mpirun -np ${PARALLEL_TESTS} $SPDYN"
./test_rpath.py     "mpirun -np ${PARALLEL_TESTS} $SPDYN"
./test_gamd.py      "mpirun -np ${PARALLEL_TESTS} $SPDYN"
./test_nonstrict.py "mpirun -np ${PARALLEL_TESTS} $SPDYN" # ?

Notes

  • (Please also check notes of CPU version.)
  • When CUDA 12.0 is employed, obsolete functions such as __shfl_xor cause compilation error.
  • (Jan 15, 2024) Fix: specification of compute capability 8.0 was not used in the actual build.
View PDF