LAMMPS 29Sep21 Update 3 with GPU support (intel MPI)

ウェブページ

https://www.lammps.org

バージョン

29Sep21 Update 3

ビルド環境

GCC 11.2.1 (gcc-toolset-11)
Intel MKL 2023.1.0
Intel MPI 2021.9.0
CUDA 11.6

ビルドに必要なファイル

lammps-stable_29Sep2021_update3.tar.gz
(一部ファイルは以下スクリプト中で取得)

ビルド手順

#!/bin/sh

VERSION=29Sep21
NAME=lammps-stable_29Sep2021_update3
INSTALL_PREFIX=/apl/lammps/2021-Sep29-impi-CUDA

BASEDIR=/home/users/${USER}/Software/LAMMPS/${VERSION}
LAMMPS_TARBALL=${BASEDIR}/${NAME}.tar.gz

WORKDIR=/gwork/users/${USER}/lammps-29Sep2021-cuda
LAMMPS_WORKDIR=${WORKDIR}/${NAME}

GPU_ARCH=sm_80
VMD_MOLFILE_INC=/home/users/${USER}/Software/VMD/1.9.4/vmd-1.9.4a57/plugins/include

PARALLEL=12

#------------------------------------------------------------------
umask 0022
export LANG=C
ulimit -s unlimited

module -s purge
module -s load gcc-toolset/11
module -s load intelmpi/2021.9.0
module -s load mkl/2023.1.0
module -s load cuda/11.6

export CC=mpicc
export CXX=mpicxx
export FC=mpif90
export MPICC=mpicc
export MPICXX=mpicxx
export MPIFC=mpif90

PYTHONEXE=/usr/bin/python3.6m
PYTHONINC=/usr/include/python3.6m
PYTHONLIB=/usr/lib64/libpython3.6m.so

cd ${WORKDIR}
if [ -d ${NAME} ]; then
mv ${NAME} lammps_erase
rm -rf lammps_erase &
fi

tar zxf ${LAMMPS_TARBALL}

cd ${NAME}
mkdir build && cd build

# Disabled PKGs:
# FFMPEG, ADIOS, MDI, VTK: noavail
# MSCG: gsl too old
# MESSAGE: ZeroMQ support not enabled
# QUIP: failed to build
# ML-HDNNP: failed to build
# KIM: CDDL is imcompatible with GPL
# LATTE: technical problem of cmake? (LAPACK and BLAS)
# NETCDF: to avoid EVP_KDF_ctrl error
# MPIIO: not maintained?

cmake ../cmake \
-DLAMMPS_MACHINE=rccs-cuda \
-DENABLE_TESTING=on \
-DCMAKE_INSTALL_PREFIX=${INSTALL_PREFIX} \
-DCMAKE_C_COMPILER=mpicc \
-DCMAKE_CXX_COMPILER=mpicxx \
-DCMAKE_Fortran_COMPILER=mpif90 \
-DCMAKE_MPI_C_COMPILER=mpicc \
-DCMAKE_MPI_CXX_COMPILER=mpicxx \
-DCMAKE_MPI_Fortran_COMPILER=mpif90 \
-DPython_EXECUTABLE=${PYTHONEXE} \
-DPython_INCLUDE_DIR=${PYTHONINC} \
-DPython_LIBRARY=${PYTHONLIB} \
-DLAMMPS_EXCEPTIONS=on \
-DBUILD_SHARED_LIBS=on \
-DBUILD_TOOLS=on \
-DBUILD_MPI=on \
-DBUILD_OMP=on \
-DFFT=MKL \
-DFFT_SINGLE=on \
-DFFT_MKL_THREADS=on \
-DWITH_JPEG=yes \
-DWITH_PNG=yes \
-DWITH_GZIP=yes \
-DPKG_ASPHERE=on \
-DPKG_ATC=on \
-DPKG_AWPMD=on \
-DPKG_BOCS=on \
-DPKG_BODY=on \
-DPKG_BROWNIAN=on \
-DPKG_CG-DNA=on \
-DPKG_CG-SDK=on \
-DPKG_CLASS2=on \
-DPKG_COLLOID=on \
-DPKG_COLVARS=on \
-DPKG_COMPRESS=on \
-DPKG_CORESHELL=on \
-DPKG_DIELECTRIC=on \
-DPKG_DIFFRACTION=on \
-DPKG_DIPOLE=on \
-DPKG_DPD-BASIC=on \
-DPKG_DPD-MESO=on \
-DPKG_DPD-REACT=on \
-DPKG_DPD-SMOOTH=on \
-DPKG_DRUDE=on \
-DPKG_EFF=on \
-DPKG_EXTRA-COMPUTE=on \
-DPKG_EXTRA-DUMP=on \
-DPKG_EXTRA-FIX=on \
-DPKG_EXTRA-MOLECULE=on \
-DPKG_EXTRA-PAIR=on \
-DPKG_FEP=on \
-DPKG_GPU=on \
-DGPU_API=cuda \
-DGPU_ARCH=${GPU_ARCH} \
-DPKG_GRANULAR=on \
-DPKG_H5MD=on \
-DPKG_INTEL=on \
-DPKG_INTERLAYER=on \
-DPKG_KIM=off \
-DDOWNLOAD_KIM=no \
-DPKG_KOKKOS=on \
-DKokkos_ARCH_ZEN3=yes \
-DKokkos_ENABLE_OPENMP=yes \
-DPKG_KSPACE=on \
-DPKG_LATBOLTZ=on \
-DPKG_MACHDYN=on \
-DDOWNLOAD_EIGEN3=on \
-DPKG_MANIFOLD=on \
-DPKG_MANYBODY=on \
-DPKG_MC=on \
-DPKG_MDI=off \
-DPKG_MEAM=on \
-DPKG_MESONT=on \
-DPKG_MESSAGE=on \
-DPKG_MGPT=on \
-DPKG_MISC=on \
-DPKG_ML-HDNNP=off \
-DDOWNLOAD_N2P2=no \
-DPKG_ML-IAP=off \
-DPKG_ML-PACE=on \
-DPKG_ML-QUIP=off \
-DDOWNLOAD_QUIP=no \
-DPKG_ML-RANN=on \
-DPKG_ML-SNAP=on \
-DPKG_MOFFF=on \
-DPKG_MOLECULE=on \
-DPKG_MOLFILE=on \
-DMOLFILE_INCLUDE_DIR=${VMD_MOLFILE_INC} \
-DPKG_MPIIO=off \
-DPKG_MSCG=off \
-DPKG_NETCDF=on \
-DPKG_OPENMP=on \
-DPKG_OPT=on \
-DPKG_ORIENT=on \
-DPKG_PERI=on \
-DPKG_PHONON=on \
-DPKG_PLUGIN=on \
-DPKG_PLUMED=on \
-DDOWNLOAD_PLUMED=yes \
-DPKG_POEMS=on \
-DPKG_PTM=on \
-DPKG_PYTHON=on \
-DPKG_QEQ=on \
-DPKG_QMMM=on \
-DPKG_QTB=on \
-DPKG_REACTION=on \
-DPKG_REAXFF=on \
-DPKG_REPLICA=on \
-DPKG_RIGID=on \
-DPKG_SCAFACOS=on \
-DDOWNLOAD_SCAFACOS=yes \
-DPKG_SHOCK=on \
-DPKG_SMTBQ=on \
-DPKG_SPH=on \
-DPKG_SPIN=on \
-DPKG_SRD=on \
-DPKG_TALLY=on \
-DPKG_UEF=on \
-DPKG_VORONOI=on \
-DDOWNLOAD_VORO=yes \
-DPKG_VTK=off \
-DPKG_YAFF=on \
-DBLAS_LIBRARIES="-Wl,--no-as-needed -lmkl_intel_lp64 -lmkl_gnu_thread -lmkl_core -lgomp -lpthread -lm -ldl" \
-DCMAKE_BUILD_TYPE=Release

make VERBOSE=1 -j ${PARALLEL}
make install

cp -a ../examples ${INSTALL_PREFIX}

cd ${INSTALL_PREFIX}
for f in etc/profile.d/*; do
ln -s $f .
done

cd lib64
if [ -f liblammps_rccs-cuda.so ]; then
ln -s liblammps_rccs-cuda.so liblammps.so
fi
if [ -f liblammps_rccs-cuda.so.0 ]; then
ln -s liblammps_rccs-cuda.so.0 liblammps.so.0
fi

テスト

#!/bin/sh
#PBS -l select=1:ncpus=16:mpiprocs=8:ompthreads=2:ngpus=2
#PBS -l walltime=24:00:00

NAME=lammps-stable_29Sep2021_update3
WORKDIR=/gwork/users/${USER}/lammps-29Sep2021-cuda
LAMMPS_WORKDIR=${WORKDIR}/${NAME}

#------------------------------------------------------------------
umask 0022
export LANG=C
ulimit -s unlimited

module -s purge
module -s load gcc-toolset/11
module -s load intelmpi/2021.9.0
module -s load mkl/2023.1.0
module -s load cuda/11.6

export CC=mpicc
export CXX=mpicxx
export FC=mpif90
export MPICC=mpicc
export MPICXX=mpicxx
export MPIFC=mpif90

cd ${LAMMPS_WORKDIR}
cd build

make test # there may be errors...

テスト結果

The following tests FAILED:
32 - SimpleCommands (SEGFAULT)
66 - MolPairStyle:born_coul_long_cs (SEGFAULT)
70 - MolPairStyle:buck (SEGFAULT)
71 - MolPairStyle:buck_coul_cut (SEGFAULT)
72 - MolPairStyle:buck_coul_long (SEGFAULT)
73 - MolPairStyle:buck_coul_long_cs (SEGFAULT)
77 - MolPairStyle:buck_long_coul_off (SEGFAULT)
78 - MolPairStyle:buck_long_cut_coul_long (SEGFAULT)
79 - MolPairStyle:buck_mdf (SEGFAULT)
81 - MolPairStyle:buck_table_coul_off (SEGFAULT)
84 - MolPairStyle:coul_cut (SEGFAULT)
88 - MolPairStyle:coul_diel (SEGFAULT)
89 - MolPairStyle:coul_dsf (SEGFAULT)
90 - MolPairStyle:coul_exclude (SEGFAULT)
97 - MolPairStyle:coul_streitz_long (SEGFAULT)
100 - MolPairStyle:coul_table_cs (SEGFAULT)
107 - MolPairStyle:gauss_cut (SEGFAULT)
111 - MolPairStyle:hybrid_multiple (SEGFAULT)
112 - MolPairStyle:lennard_mdf (SEGFAULT)
114 - MolPairStyle:lj_charmm_coul_charmm (SEGFAULT)
116 - MolPairStyle:lj_charmm_coul_long (SEGFAULT)
118 - MolPairStyle:lj_charmm_coul_table (SEGFAULT)
120 - MolPairStyle:lj_charmmfsw_coul_long (SEGFAULT)
121 - MolPairStyle:lj_charmmfsw_coul_table (SEGFAULT)
132 - MolPairStyle:lj_cut (SEGFAULT)
137 - MolPairStyle:lj_cut_coul_long (SEGFAULT)
140 - MolPairStyle:lj_cut_coul_table (SEGFAULT)
147 - MolPairStyle:lj_cut_tip4p_long (SEGFAULT)
150 - MolPairStyle:lj_expand (SEGFAULT)
152 - MolPairStyle:lj_expand_coul_table (SEGFAULT)
155 - MolPairStyle:lj_long_coul_long (SEGFAULT)
163 - MolPairStyle:lj_sdk (SEGFAULT)
173 - MolPairStyle:lj_table_coul_table (SEGFAULT)
175 - MolPairStyle:lj_table_tip4p_table (SEGFAULT)
178 - MolPairStyle:mm3_switch3_coulgauss_table (SEGFAULT)
182 - MolPairStyle:nm_cut (SEGFAULT)
186 - MolPairStyle:python_hybrid (SEGFAULT)
188 - MolPairStyle:soft (SEGFAULT)
191 - MolPairStyle:tip4p_long (SEGFAULT)
193 - MolPairStyle:tip4p_table (SEGFAULT)
199 - AtomicPairStyle:adp (SEGFAULT)
203 - AtomicPairStyle:buck_coul_cut_qeq_point (Failed)
204 - AtomicPairStyle:buck_coul_cut_qeq_shielded (Failed)
206 - AtomicPairStyle:colloid_multi (SEGFAULT)
207 - AtomicPairStyle:colloid_multi_tri (SEGFAULT)
213 - AtomicPairStyle:eam_cd (SEGFAULT)
214 - AtomicPairStyle:eam_cd_old (SEGFAULT)
221 - AtomicPairStyle:edip (Failed)
222 - AtomicPairStyle:eim (SEGFAULT)
224 - AtomicPairStyle:hybrid-eam (SEGFAULT)
225 - AtomicPairStyle:hybrid-eam_fs (SEGFAULT)
229 - AtomicPairStyle:meam_spline (SEGFAULT)
230 - AtomicPairStyle:meam_sw_spline (SEGFAULT)
231 - AtomicPairStyle:momb (SEGFAULT)
233 - AtomicPairStyle:reaxff (SEGFAULT)
234 - AtomicPairStyle:reaxff_lgvdw (Failed)
235 - AtomicPairStyle:reaxff_noqeq (Failed)
236 - AtomicPairStyle:reaxff_tabulate (Failed)
238 - AtomicPairStyle:table_linear (SEGFAULT)
240 - AtomicPairStyle:table_spline (SEGFAULT)
241 - AtomicPairStyle:yukawa_colloid (SEGFAULT)
242 - ManybodyPairStyle:airebo (SEGFAULT)
243 - ManybodyPairStyle:airebo_00 (SEGFAULT)
244 - ManybodyPairStyle:airebo_m (SEGFAULT)
245 - ManybodyPairStyle:airebo_m00 (SEGFAULT)
246 - ManybodyPairStyle:bop (SEGFAULT)
247 - ManybodyPairStyle:bop_save (Failed)
250 - ManybodyPairStyle:drip (Failed)
251 - ManybodyPairStyle:drip_real (Failed)
256 - ManybodyPairStyle:ilp-graphene-hbn (SEGFAULT)
257 - ManybodyPairStyle:ilp-graphene-hbn_notaper (SEGFAULT)
258 - ManybodyPairStyle:kolmogorov_crespi_full (Failed)
259 - ManybodyPairStyle:kolmogorov_crespi_full_notaper (SEGFAULT)
262 - ManybodyPairStyle:lebedeva_z (Failed)
263 - ManybodyPairStyle:meam (SEGFAULT)
275 - ManybodyPairStyle:rann (SEGFAULT)
277 - ManybodyPairStyle:snap (SEGFAULT)
290 - ManybodyPairStyle:vashishta (SEGFAULT)
291 - ManybodyPairStyle:vashishta_table (SEGFAULT)
331 - KSpaceStyle:ewald_disp (SEGFAULT)
333 - KSpaceStyle:ewald_nozforce (SEGFAULT)
336 - KSpaceStyle:ewald_tri (SEGFAULT)
337 - KSpaceStyle:pppm (SEGFAULT)
338 - KSpaceStyle:pppm_ad (SEGFAULT)
339 - KSpaceStyle:pppm_cg (SEGFAULT)
340 - KSpaceStyle:pppm_cg_ad (SEGFAULT)
341 - KSpaceStyle:pppm_cg_tiled (SEGFAULT)
342 - KSpaceStyle:pppm_dipole (SEGFAULT)
347 - KSpaceStyle:pppm_disp_tip4p (Failed)
349 - KSpaceStyle:pppm_slab (SEGFAULT)
354 - KSpaceStyle:pppm_tip4p (Failed)
357 - KSpaceStyle:pppm_tip4p_slab (SEGFAULT)
358 - KSpaceStyle:pppm_tri (SEGFAULT)
359 - KSpaceStyle:scafacos_direct (Failed)
360 - KSpaceStyle:scafacos_ewald (Failed)
361 - KSpaceStyle:scafacos_fmm (Failed)
362 - KSpaceStyle:scafacos_fmm_tuned (Failed)
363 - KSpaceStyle:scafacos_p2nfft (Failed)
364 - FixTimestep:adapt_coul (Failed)
381 - FixTimestep:momentum (Failed)
453 - DihedralStyle:quadratic (Failed)
455 - DihedralStyle:table_cut_linear (Failed)
457 - DihedralStyle:table_linear (Failed)
458 - DihedralStyle:table_spline (Failed)

メモ

CPU版と2022-Jun23 intelmpi GPU版のメモを参考にしてください。

View PDF