Commit e58bcd8b authored by sjplimp's avatar sjplimp Committed by GitHub
Browse files

Merge pull request #568 from akohlmey/fix_gcmc_parallel_workaround

Workaround for data corruption when using fix gcmc in parallel with shake or fix rigid
parents ef2f4980 c9a0d38a
Loading
Loading
Loading
Loading
+3 −0
Original line number Diff line number Diff line
@@ -383,6 +383,9 @@ called. Reneighboring is required.
Can be run in parallel, but aspects of the GCMC part will not scale
well in parallel. Only usable for 3D simulations.

When using fix gcmc in combination with fix shake or fix rigid,
only gcmc exchange moves are supported.

Note that very lengthy simulations involving insertions/deletions of
billions of gas molecules may run out of atom or molecule IDs and
trigger an error, so it is better to run multiple shorter-duration

examples/gcmc/H2O.txt

0 → 100644
+62 −0
Original line number Diff line number Diff line
# CO2 molecule file. TraPPE model.

3 atoms
2 bonds
1 angles

Coords

1    1.12456   0.09298   1.27452
2    1.53683   0.75606   1.89928
3    0.49482   0.56390   0.65678

Types

1        1  
2        2   
3        2  

Charges 

1       -0.8472
2        0.4236 
3        0.4236  

Bonds

1   1      1      2
2   1      1      3

Angles

1   1      2      1      3

Shake Flags

1 1
2 1
3 1

Shake Atoms

1 1 2 3
2 1 2 3
3 1 2 3

Shake Bond Types

1 1 1 1
2 1 1 1
3 1 1 1

Special Bond Counts

1 2 0 0
2 1 1 0
3 1 1 0

Special Bonds

1 2 3
2 1 3
3 1 2
+1 −1
Original line number Diff line number Diff line
@@ -64,7 +64,7 @@ fix_modify myrigidnvt dynamic/dof no
# gcmc

variable        tfac equal 5.0/3.0 # (3 trans + 2 rot)/(3 trans)
fix             mygcmc all gcmc 100 100 100 0 54341 ${temp} ${mu} ${disp} mol &
fix             mygcmc all gcmc 100 100 0 0 54341 ${temp} ${mu} ${disp} mol &
                co2mol tfac_insert ${tfac} group co2 rigid myrigidnvt

# output
+88 −0
Original line number Diff line number Diff line
# fix gcmc example with fix shake

# variables available on command line

variable        mu index -8.1
variable	disp index 0.5
variable        temp index 338.0
variable        lbox index 10.0
variable        spacing index 5.0

# global model settings

units           real
atom_style      full
boundary        p p p
pair_style      lj/cut/coul/long  14 
pair_modify     mix arithmetic tail yes
kspace_style    ewald 0.0001
bond_style      harmonic
angle_style     harmonic

# box, start molecules on simple cubic lattice

lattice 	sc ${spacing}
region          box block 0 ${lbox} 0 ${lbox} 0 ${lbox} units box
create_box      2 box                       &
                bond/types 1                &
                angle/types 1               &
                extra/bond/per/atom 2       &
                extra/angle/per/atom 1      &
                extra/special/per/atom 2

# we can load multiple molecule templates, but don't have to use them all
molecule        co2mol CO2.txt
molecule        h2omol H2O.txt
create_atoms   	0 box mol h2omol 464563 units box
                        
# rigid SPC/E water model

pair_coeff      1 1 0.15535 3.166
pair_coeff      * 2 0.0000 0.0000       

bond_coeff      1     1000       1.0   
angle_coeff     1      100       109.47

# masses

mass 1 15.9994 
mass 2 1.0

# MD settings

group           h2o type 1 2
neighbor        2.0 bin
neigh_modify    every 1 delay 1 check yes
velocity       	all create ${temp} 54654
timestep        1.0

minimize 0.0 0.0 100 1000
reset_timestep 0
# rigid constraints with thermostat 

fix             mynvt all nvt temp ${temp} ${temp} 100
fix             wshake  all shake 0.0001 50 0 b 1 a 1 mol h2omol
# gcmc



run 1000

variable        tfac equal 5.0/3.0 # (3 trans + 2 rot)/(3 trans)
fix             mygcmc all gcmc 100 100 0 0 54341 ${temp} ${mu} ${disp} mol &
                h2omol tfac_insert ${tfac} group h2o shake wshake

# output

variable	tacc equal f_mygcmc[2]/(f_mygcmc[1]+0.1)
variable	iacc equal f_mygcmc[4]/(f_mygcmc[3]+0.1)
variable	dacc equal f_mygcmc[6]/(f_mygcmc[5]+0.1)
variable	racc equal f_mygcmc[8]/(f_mygcmc[7]+0.1)
compute_modify  thermo_temp dynamic/dof yes
thermo_style    custom step temp press pe ke density atoms v_iacc v_dacc v_tacc v_racc
thermo          1000

# run

run             20000
+51 −48
Original line number Diff line number Diff line
LAMMPS (17 Mar 2017)
LAMMPS (6 Jul 2017)
  using 1 OpenMP thread(s) per MPI task
# GCMC for CO2 molecular fluid, rigid/small/nvt dynamics
# Rigid CO2 TraPPE model
# [Potoff and J.I. Siepmann, Vapor-liquid equilibria of
@@ -80,11 +81,11 @@ fix_modify myrigidnvt dynamic/dof no
# gcmc

variable        tfac equal 5.0/3.0 # (3 trans + 2 rot)/(3 trans)
fix             mygcmc all gcmc 100 100 100 0 54341 ${temp} ${mu} ${disp} mol                 co2mol tfac_insert ${tfac} group co2 rigid myrigidnvt
fix             mygcmc all gcmc 100 100 100 0 54341 338.0 ${mu} ${disp} mol                 co2mol tfac_insert ${tfac} group co2 rigid myrigidnvt
fix             mygcmc all gcmc 100 100 100 0 54341 338.0 -8.1 ${disp} mol                 co2mol tfac_insert ${tfac} group co2 rigid myrigidnvt
fix             mygcmc all gcmc 100 100 100 0 54341 338.0 -8.1 0.5 mol                 co2mol tfac_insert ${tfac} group co2 rigid myrigidnvt
fix             mygcmc all gcmc 100 100 100 0 54341 338.0 -8.1 0.5 mol                 co2mol tfac_insert 1.66666666666667 group co2 rigid myrigidnvt
fix             mygcmc all gcmc 100 100 0 0 54341 ${temp} ${mu} ${disp} mol                 co2mol tfac_insert ${tfac} group co2 rigid myrigidnvt
fix             mygcmc all gcmc 100 100 0 0 54341 338.0 ${mu} ${disp} mol                 co2mol tfac_insert ${tfac} group co2 rigid myrigidnvt
fix             mygcmc all gcmc 100 100 0 0 54341 338.0 -8.1 ${disp} mol                 co2mol tfac_insert ${tfac} group co2 rigid myrigidnvt
fix             mygcmc all gcmc 100 100 0 0 54341 338.0 -8.1 0.5 mol                 co2mol tfac_insert ${tfac} group co2 rigid myrigidnvt
fix             mygcmc all gcmc 100 100 0 0 54341 338.0 -8.1 0.5 mol                 co2mol tfac_insert 1.66666666666667 group co2 rigid myrigidnvt

# output

@@ -106,7 +107,7 @@ WARNING: Using 12-bit tables for long-range coulomb (../kspace.cpp:321)
  estimated relative force accuracy = 0.0001
  KSpace vectors: actual max1d max3d = 16 2 62
                  kxmax kymax kzmax  = 2 2 2
WARNING: Fix gcmc using full_energy option (../fix_gcmc.cpp:439)
WARNING: Fix gcmc using full_energy option (../fix_gcmc.cpp:445)
0 atoms in group FixGCMC:gcmc_exclusion_group:mygcmc
0 atoms in group FixGCMC:rotation_gas_atoms:mygcmc
WARNING: Neighbor exclusions used with KSpace solver may give inconsistent Coulombic energies (../neighbor.cpp:472)
@@ -122,56 +123,58 @@ Neighbor list info ...
      pair build: half/bin/newton
      stencil: half/bin/3d/newton
      bin: standard
Per MPI rank memory allocation (min/avg/max) = 15.61 | 15.61 | 15.61 Mbytes
Per MPI rank memory allocation (min/avg/max) = 15.62 | 15.62 | 15.62 Mbytes
Step Temp Press PotEng KinEng Density Atoms v_iacc v_dacc v_tacc v_racc 
       0    364.27579    4238.8631   -9.6809388    13.391989    0.5846359       24            0            0            0            0 
    1000    311.39835   -327.93481   -8.6795381    9.9010062   0.51155641       21   0.13302848   0.12331626    0.6894397   0.90997852 
WARNING: Using kspace solver on system with no charge (../kspace.cpp:289)
    2000    905.66812    319.43347  -0.50350961    6.2991241   0.14615898        6   0.20952183   0.20430213   0.71797992   0.92626683 
    3000    275.57393   -719.89718   -26.534978    14.238181   0.80387436       33   0.21291069   0.20460696   0.72899202    0.9133259 
    4000    254.70771   -245.01902   -20.981537    13.160079   0.80387436       33   0.17245726   0.16974613   0.70145764   0.90542759 
    5000    96.073601   -517.98124   -34.019065     5.441166   0.87695385       36   0.14174575   0.13607057    0.6776754   0.90155771 
    6000    397.57265    148.92645   -7.2012893    10.665797   0.43847693       18   0.12299956    0.1202471   0.66165464   0.90274793 
    7000     455.4271   -347.44181   -5.9244703    12.217875   0.43847693       18   0.15182038   0.14791307   0.67904236   0.90560829 
    8000    301.03124   -627.45324   -13.251012    11.066909    0.5846359       24   0.16687346   0.16315516    0.6936719   0.91129375 
    9000     256.5747   -565.67983   -17.814128    11.981874   0.73079488       30   0.15458482   0.15131825   0.68966283   0.90993975 
   10000    443.60076    89.586912    -6.077863    11.900606   0.43847693       18   0.16092552   0.16020353   0.69882461   0.91422145 
   11000    436.43777    64.412921   -6.7128469    11.708443   0.43847693       18   0.17453966   0.17480683   0.70679243   0.91369445 
   12000    594.42207    849.07743   -3.3708621    10.040536   0.29231795       12   0.17461606   0.17568622   0.71175869   0.91333367 
   13000    426.85849   -1093.1334   -17.524618    17.813377   0.65771539       27   0.17742896   0.17792831   0.71363306   0.91450124 
   14000    317.75995    336.31107    -10.46774    11.681912    0.5846359       24   0.18331181   0.18427921   0.71715557   0.91652256 
   15000    272.65129    317.50536   -26.428336    14.087176   0.80387436       33   0.17449167     0.175957   0.71122398   0.91528038 
   16000    344.28567   -577.91079   -18.177927    16.077919   0.73079488       30    0.1661682   0.16781514   0.70485136   0.91508882 
   17000    134.55928    -193.5668   -30.297136    7.6208177   0.87695385       36   0.15965609    0.1605036   0.69658104    0.9140445 
   18000    231.87302   -446.07671   -14.875027    9.6763722   0.65771539       27   0.15270985   0.15351831   0.69002918   0.91372795 
   19000     328.6835   -280.22365   -20.001303    16.982214   0.80387436       33   0.15201017   0.15272181   0.69023195   0.91272534 
   20000            0    -20.39554  -0.14872889           -0            0        0   0.15600204   0.15750795   0.69503275    0.9138765 
Loop time of 30.9008 on 1 procs for 20000 steps with 0 atoms

Performance: 55.921 ns/day, 0.429 hours/ns, 647.233 timesteps/s
99.8% CPU use with 1 MPI tasks x no OpenMP threads
    1000    420.43475    1722.4052   -9.6956123    15.456579    0.5846359       24   0.20879341   0.20713005            0            0 
    2000    302.29516   -547.83641   -22.017674     14.11699   0.73079488       30    0.1742478    0.1678018            0            0 
    3000     316.6934   -1080.2672   -8.2218891    10.069364   0.51155641       21   0.13544917   0.13720634            0            0 
    4000    246.81618   -679.83642   -14.577244     10.29997   0.65771539       27    0.1568939   0.15860229            0            0 
    5000    260.22849   -896.29914   -16.097593    10.859684   0.65771539       27   0.13138744   0.13547049            0            0 
    6000    291.70796     -1521.99   -22.303136    13.622574   0.73079488       30   0.12615476   0.12717694            0            0 
    7000    236.02638   -599.92186   -27.580831    13.367447   0.87695385       36     0.119703   0.12145398            0            0 
    8000    321.45341    688.10577    -10.09204    11.817696    0.5846359       24   0.10917411   0.11032646            0            0 
    9000    502.85382   -302.31056  -0.22330142   0.99927447  0.073079488        3    0.1254105   0.12905828            0            0 
   10000    249.98239    -510.0091   -32.815145    15.399767   0.95003334       39    0.1274504   0.12875623            0            0 
   11000    247.59424   -1129.0274   -25.320205    12.792544   0.80387436       33   0.11739076   0.11916784            0            0 
   12000            0    -20.39554  -0.14872889           -0            0        0    0.1254933   0.12920375            0            0 
   13000    1272.2738   -474.79484  -0.29450485    8.8489483   0.14615898        6   0.13767133   0.14112496            0            0 
   14000    516.54246   -36.296516   -5.0012009    11.291243   0.36539744       15   0.15632744   0.15955377            0            0 
   15000    307.09233    1951.9301   -14.820362    12.815375   0.65771539       27   0.15393544   0.15716192            0            0 
   16000    198.31989   -559.48443   -30.459487    11.231925   0.87695385       36    0.1482565   0.15025652            0            0 
   17000    246.99311    657.85683   -18.579206     11.53442   0.73079488       30   0.14143958   0.14375423            0            0 
   18000    467.13468    167.03738   -1.0945268     5.569759   0.21923846        9   0.13847359   0.14098533            0            0 
   19000    359.54027   -1413.5407   -12.156233    13.217895    0.5846359       24   0.15169146   0.15294205            0            0 
   20000    227.79597   -1204.5652    -23.24144    10.637925   0.73079488       30   0.14917199   0.15022946            0            0 
Loop time of 20.153 on 1 procs for 20000 steps with 30 atoms

Performance: 85.744 ns/day, 0.280 hours/ns, 992.408 timesteps/s
99.3% CPU use with 1 MPI tasks x 1 OpenMP threads

MPI task timing breakdown:
Section |  min time  |  avg time  |  max time  |%varavg| %total
---------------------------------------------------------------
Pair    | 2.1985     | 2.1985     | 2.1985     |   0.0 |  7.11
Bond    | 0.029596   | 0.029596   | 0.029596   |   0.0 |  0.10
Kspace  | 0.23123    | 0.23123    | 0.23123    |   0.0 |  0.75
Neigh   | 0.16141    | 0.16141    | 0.16141    |   0.0 |  0.52
Comm    | 0.20628    | 0.20628    | 0.20628    |   0.0 |  0.67
Output  | 0.00068831 | 0.00068831 | 0.00068831 |   0.0 |  0.00
Modify  | 28.022     | 28.022     | 28.022     |   0.0 | 90.69
Other   |            | 0.05058    |            |       |  0.16

Nlocal:    0 ave 0 max 0 min
Pair    | 2.5352     | 2.5352     | 2.5352     |   0.0 | 12.58
Bond    | 0.026112   | 0.026112   | 0.026112   |   0.0 |  0.13
Kspace  | 0.25       | 0.25       | 0.25       |   0.0 |  1.24
Neigh   | 0.10364    | 0.10364    | 0.10364    |   0.0 |  0.51
Comm    | 0.22907    | 0.22907    | 0.22907    |   0.0 |  1.14
Output  | 0.0013065  | 0.0013065  | 0.0013065  |   0.0 |  0.01
Modify  | 16.957     | 16.957     | 16.957     |   0.0 | 84.14
Other   |            | 0.05061    |            |       |  0.25

Nlocal:    30 ave 30 max 30 min
Histogram: 1 0 0 0 0 0 0 0 0 0
Nghost:    0 ave 0 max 0 min
Nghost:    2310 ave 2310 max 2310 min
Histogram: 1 0 0 0 0 0 0 0 0 0
Neighs:    0 ave 0 max 0 min
Neighs:    7736 ave 7736 max 7736 min
Histogram: 1 0 0 0 0 0 0 0 0 0

Total # of neighbors = 0
Neighbor list builds = 40367
Dangerous builds = 118
Total # of neighbors = 7736
Ave neighs/atom = 257.867
Ave special neighs/atom = 2
Neighbor list builds = 20349
Dangerous builds = 0

Total wall time: 0:00:30
Total wall time: 0:00:20
Loading