Commit ea4d11fc authored by Stan Moore's avatar Stan Moore
Browse files

Merge branch 'master' into USER-DPD_kokkos_testing

parents 67a0183b 1370385c

doc/src/JPG/user_intel.png

−197 B (14.1 KiB)
Loading image diff...

doc/src/Manual.txt

+2 −2
Original line number Diff line number Diff line 
<!-- HTML_ONLY -->

<HEAD>

<TITLE>LAMMPS Users Manual</TITLE>

<META NAME="docnumber" CONTENT="19 May 2017 version">

<META NAME="docnumber" CONTENT="23 Jun 2017 version">

<META NAME="author" CONTENT="http://lammps.sandia.gov - Sandia National Laboratories">

<META NAME="copyright" CONTENT="Copyright (2003) Sandia Corporation.  This software and manual is distributed under the GNU General Public License.">

</HEAD>
@@ -21,7 +21,7 @@ 
<H1></H1>



LAMMPS Documentation :c,h3

19 May 2017 version :c,h4

23 Jun 2017 version :c,h4



Version info: :h4

doc/src/Section_commands.txt

+3 −3
Original line number Diff line number Diff line
@@ -964,7 +964,7 @@ KOKKOS, o = USER-OMP, t = OPT. 
"lj/expand (gko)"_pair_lj_expand.html,

"lj/gromacs (gko)"_pair_gromacs.html,

"lj/gromacs/coul/gromacs (ko)"_pair_gromacs.html,

"lj/long/coul/long (o)"_pair_lj_long.html,

"lj/long/coul/long (io)"_pair_lj_long.html,

"lj/long/dipole/long"_pair_dipole.html,

"lj/long/tip4p/long"_pair_lj_long.html,

"lj/smooth (o)"_pair_lj_smooth.html,
@@ -1225,7 +1225,7 @@ USER-OMP, t = OPT. 
"msm/cg (o)"_kspace_style.html,

"pppm (go)"_kspace_style.html,

"pppm/cg (o)"_kspace_style.html,

"pppm/disp"_kspace_style.html,

"pppm/disp (i)"_kspace_style.html,

"pppm/disp/tip4p"_kspace_style.html,

"pppm/stagger"_kspace_style.html,

"pppm/tip4p (o)"_kspace_style.html :tb(c=4,ea=c)

doc/src/accelerate_intel.txt

+73 −39
Original line number Diff line number Diff line
@@ -30,8 +30,8 @@ Dihedral Styles: charmm, harmonic, opls :l 
Fixes: nve, npt, nvt, nvt/sllod :l

Improper Styles: cvff, harmonic :l

Pair Styles: buck/coul/cut, buck/coul/long, buck, eam, gayberne,

charmm/coul/long, lj/cut, lj/cut/coul/long, sw, tersoff :l

K-Space Styles: pppm :l

charmm/coul/long, lj/cut, lj/cut/coul/long, lj/long/coul/long, sw, tersoff :l

K-Space Styles: pppm, pppm/disp :l

:ule



[Speed-ups to expect:]
@@ -42,62 +42,88 @@ precision mode. Performance improvements are shown compared to 
LAMMPS {without using other acceleration packages} as these are

under active development (and subject to performance changes). The

measurements were performed using the input files available in

the src/USER-INTEL/TEST directory. These are scalable in size; the

results given are with 512K particles (524K for Liquid Crystal).

Most of the simulations are standard LAMMPS benchmarks (indicated

by the filename extension in parenthesis) with modifications to the

run length and to add a warmup run (for use with offload

benchmarks).

the src/USER-INTEL/TEST directory with the provided run script.

These are scalable in size; the results given are with 512K

particles (524K for Liquid Crystal). Most of the simulations are

standard LAMMPS benchmarks (indicated by the filename extension in

parenthesis) with modifications to the run length and to add a

warmup run (for use with offload benchmarks).



:c,image(JPG/user_intel.png)



Results are speedups obtained on Intel Xeon E5-2697v4 processors

(code-named Broadwell) and Intel Xeon Phi 7250 processors

(code-named Knights Landing) with "18 Jun 2016" LAMMPS built with

Intel Parallel Studio 2016 update 3. Results are with 1 MPI task

(code-named Knights Landing) with "June 2017" LAMMPS built with

Intel Parallel Studio 2017 update 2. Results are with 1 MPI task

per physical core. See {src/USER-INTEL/TEST/README} for the raw

simulation rates and instructions to reproduce.



:line



[Accuracy and order of operations:]



In most molecular dynamics software, parallelization parameters

(# of MPI, OpenMP, and vectorization) can change the results due

to changing the order of operations with finite-precision

calculations. The USER-INTEL package is deterministic. This means

that the results should be reproducible from run to run with the

{same} parallel configurations and when using determinstic

libraries or library settings (MPI, OpenMP, FFT). However, there

are differences in the USER-INTEL package that can change the

order of operations compared to LAMMPS without acceleration:



Neighbor lists can be created in a different order :ulb,l

Bins used for sorting atoms can be oriented differently :l

The default stencil order for PPPM is 7. By default, LAMMPS will

calculate other PPPM parameters to fit the desired acuracy with

this order :l

The {newton} setting applies to all atoms, not just atoms shared

between MPI tasks :l

Vectorization can change the order for adding pairwise forces :l

:ule



The precision mode (described below) used with the USER-INTEL

package can change the {accuracy} of the calculations. For the

default {mixed} precision option, calculations between pairs or

triplets of atoms are performed in single precision, intended to

be within the inherent error of MD simulations. All accumulation

is performed in double precision to prevent the error from growing

with the number of atoms in the simulation. {Single} precision

mode should not be used without appropriate validation.



:line



[Quick Start for Experienced Users:]



LAMMPS should be built with the USER-INTEL package installed.

Simulations should be run with 1 MPI task per physical {core},

not {hardware thread}.



For Intel Xeon CPUs:



Edit src/MAKE/OPTIONS/Makefile.intel_cpu_intelmpi as necessary. :ulb,l

If using {kspace_style pppm} in the input script, add "neigh_modify binsize cutoff" and "kspace_modify diff ad" to the input script for better

performance.  Cutoff should be roughly the neighbor list cutoff.  By

default the binsize is half the neighbor list cutoff.  :l

"-pk intel 0 omp 2 -sf intel" added to LAMMPS command-line :l

Set the environment variable KMP_BLOCKTIME=0 :l

"-pk intel 0 omp $t -sf intel" added to LAMMPS command-line :l

$t should be 2 for Intel Xeon CPUs and 2 or 4 for Intel Xeon Phi :l

For some of the simple 2-body potentials without long-range

electrostatics, performance and scalability can be better with

the "newton off" setting added to the input script :l

If using {kspace_style pppm} in the input script, add

"kspace_modify diff ad" for better performance :l

:ule



For Intel Xeon Phi CPUs for simulations without {kspace_style

pppm} in the input script :

For Intel Xeon Phi CPUs:



Edit src/MAKE/OPTIONS/Makefile.knl as necessary. :ulb,l

Runs should be performed using MCDRAM. :l

"-pk intel 0 omp 2 -sf intel" {or} "-pk intel 0 omp 4 -sf intel"

should be added to the LAMMPS command-line. Choice for best

performance will depend on the simulation. :l

Runs should be performed using MCDRAM. :ulb,l

:ule



For Intel Xeon Phi CPUs for simulations with {kspace_style

pppm} in the input script:



Edit src/MAKE/OPTIONS/Makefile.knl as necessary. :ulb,l

Runs should be performed using MCDRAM. :l

Add "neigh_modify binsize 3" to the input script for better

performance. :l

Add "kspace_modify diff ad" to the input script for better

performance. :l

export KMP_AFFINITY=none :l

"-pk intel 0 omp 3 lrt yes -sf intel" or "-pk intel 0 omp 1 lrt yes

-sf intel" added to LAMMPS command-line. Choice for best performance

will depend on the simulation. :l

For simulations using {kspace_style pppm} on Intel CPUs

supporting AVX-512:



Add "kspace_modify diff ad" to the input script :ulb,l

The command-line option should be changed to

"-pk intel 0 omp $r lrt yes -sf intel" where $r is the number of

threads minus 1. :l

Do not use thread affinity (set KMP_AFFINITY=none) :l

The "newton off" setting may provide better scalability :l

:ule



For Intel Xeon Phi coprocessors (Offload):
@@ -169,6 +195,10 @@ cat /proc/cpuinfo :pre 


[Building LAMMPS with the USER-INTEL package:]



NOTE: See the src/USER-INTEL/README file for additional flags that

might be needed for best performance on Intel server processors

code-named "Skylake".



The USER-INTEL package must be installed into the source directory:



make yes-user-intel :pre
@@ -322,8 +352,8 @@ follow in the input script. 


NOTE: The USER-INTEL package will perform better with modifications

to the input script when "PPPM"_kspace_style.html is used:

"kspace_modify diff ad"_kspace_modify.html and "neigh_modify binsize

3"_neigh_modify.html should be added to the input script.

"kspace_modify diff ad"_kspace_modify.html should be added to the

input script.



Long-Range Thread (LRT) mode is an option to the "package

intel"_package.html command that can improve performance when using
@@ -342,6 +372,10 @@ would normally perform best with "-pk intel 0 omp 4", instead use 
environment variable "KMP_AFFINITY=none". LRT mode is not supported

when using offload.



NOTE: Changing the "newton"_newton.html setting to off can improve

performance and/or scalability for simple 2-body potentials such as

lj/cut or when using LRT mode on processors supporting AVX-512.



Not all styles are supported in the USER-INTEL package. You can mix

the USER-INTEL package with styles from the "OPT"_accelerate_opt.html

package or the "USER-OMP package"_accelerate_omp.html. Of course,
@@ -467,7 +501,7 @@ supported. 


Brown, W.M., Carrillo, J.-M.Y., Mishra, B., Gavhane, N., Thakker, F.M., De Kraker, A.R., Yamada, M., Ang, J.A., Plimpton, S.J., "Optimizing Classical Molecular Dynamics in LAMMPS," in Intel Xeon Phi Processor High Performance Programming: Knights Landing Edition, J. Jeffers, J. Reinders, A. Sodani, Eds. Morgan Kaufmann. :ulb,l



Brown, W. M., Semin, A., Hebenstreit, M., Khvostov, S., Raman, K., Plimpton, S.J. Increasing Molecular Dynamics Simulation Rates with an 8-Fold Increase in Electrical Power Efficiency. 2016 International Conference for High Performance Computing. In press. :l

Brown, W. M., Semin, A., Hebenstreit, M., Khvostov, S., Raman, K., Plimpton, S.J. "Increasing Molecular Dynamics Simulation Rates with an 8-Fold Increase in Electrical Power Efficiency."_http://dl.acm.org/citation.cfm?id=3014915 2016 High Performance Computing, Networking, Storage and Analysis, SC16: International Conference (pp. 82-95). :l



Brown, W.M., Carrillo, J.-M.Y., Gavhane, N., Thakkar, F.M., Plimpton, S.J. Optimizing Legacy Molecular Dynamics Software with Directive-Based Offload. Computer Physics Communications. 2015. 195: p. 95-101. :l

:ule

doc/src/compute_pair_local.txt

+3 −1
Original line number Diff line number Diff line
@@ -76,7 +76,9 @@ command for the types of the two atoms is used. For the {radius} 
setting, the sum of the radii of the two particles is used as a

cutoff.  For example, this is appropriate for granular particles which

only interact when they are overlapping, as computed by "granular pair

styles"_pair_gran.txt.

styles"_pair_gran.txt.  Note that if a granular model defines atom

types such that all particles of a specific type are monodisperse

(same diameter), then the two settings are effectively identical.



Note that as atoms migrate from processor to processor, there will be

no consistent ordering of the entries within the local vector or array

CRA Git | Maintained and supported by SUSTech CRA and CCSE