more info:

150000000 steps

PP tasks will do (non-perturbed) short-ranged and most bonded interactions on the GPU

PP task will update and constrain coordinates on the GPU

Using 8 MPI threads

Using 4 OpenMP threads per tMPI thread

Hi,

How many cores/threads your computer/server has?

I suggest you to create a system with lower number of steps (~1000 or 2000) and try different combinations of MPI, tMPI and GPUs.

Use "pin on", and try to use one GPU at a time to see what gives you better results. Try to change "-nt" (number of threads) parameters as well.

I tried this and got the same speed with 12 core and 1 gpu Vs 48 cores and 2 GPUs. Sometimes, more core/GPU can negatively affect the speed as communications overhead increases (or something like that).

Hi Zahra,

Firstly, unless specifically needed for other reasons, 300k water atoms per 3000 protein atoms seems a bit excessive, you can rethink if you indeed need that much water. The original system from the Sci Rep paper seems to have just ~60k water atoms.

Secondly, as Ayaz pointed out, Gromacs (especially if not one of the absolutely latest versions) will not leverage GPU-to-GPU communication, so throwing 2 GPUs on one simulation might actually slow it down. If you prefer to use both anyway, it's more reasonable to run two replicas with one GPU per replica. Depending on how many CPUs the node has, you can also choose where PME should be calculated (I can't see how it's being handled in your setup).

Finally, I found OpenMM to be a nice alternative that provides really good out-of-the-box performance on single GPUs and works almost seamlessly with Gromacs topologies, although not all details (such as thermostat/barostat) will be correctly reproduced.