MDBenchmark — quickly generate, start and analyze benchmarks for GROMACS simulations.

MDBenchmark is a tool to squeeze the maximum out of your limited computing resources. It tries to make it as easy as possible to set up systems on varying numbers of nodes and compare their performances to each other.

You can also create a plot to get a quick overview of the possible performance (and also show of to your friends)! The plot below shows the performance of an MD system on up to five nodes with and without GPUs.

You can install mdbenchmark via pip or conda:

$ pip install mdbenchmark

$ conda install -c conda-forge mdbenchmark

We recommend to install the package inside a conda environment. This can easily be done with conda. The following commands create an environment called benchmark and then installs mdbenchmark and its dependencies.

$ conda create -n benchmark
$ conda install -n benchmark -c conda-forge mdsynthesis click mdbenchmark

Before every usage of mdbenchmark, you need to first activate the conda environment. After doing this once, you can use the tool for the whole duration of your shell session.

$ source activate benchmark
# mdbenchmark should now be usable!
$ mdbenchmark
Usage: mdbenchmark [OPTIONS] COMMAND [ARGS]...

  Generate and run benchmark jobs for GROMACS simulations.

Options:
  --version  Show the version and exit.
  --help     Show this message and exit.

Commands:
  analyze   analyze finished benchmark.
  generate  Generate benchmark queuing jobs.
  submit    Start benchmark simulations.

• Generates benchmarks for GROMACS systems (contributions for other MD systems are welcome!).
• Automatically detects the queuing system on your high-performance cluster and submits jobs accordingly.
• Grabs performance from GROMACS logs and creates a fancy plot.
• Can benchmark systems either with or without GPUs.

After installation, the mdbenchmark command should be available to you globally. If you have installed the package in a virtual environment, make sure to activate that first!

Assuming your TPR files is called protein.tpr and want to run benchmarks with the module gromacs/5.1.4-plumed2.3 on up to ten nodes, use the following command:

$ mdbenchmark generate --name protein --module gromacs/5.1.4-plumed2.3 --max-nodes 10

To run benchmarks on GPUs add the --gpu flag:

$ mdbenchmark generate --name protein --module gromacs/5.1.4-plumed2.3 --max-nodes 10 --gpu

You can also create benchmarks for different versions of GROMACS:

$ mdbenchmark generate --name protein --module gromacs/5.1.4-plumed2.3 --module gromacs/2016.4-plumed2.3 --max-nodes 10 --gpu

After you generated all benchmark systems, you can start them at once:

$ mdbenchmark start

As soon as the benchmark simulations have been submitted you can run the analysis script via mdbenchmark analysis. As soon as at least one system has finished the script will produce a .csv output file or a plot for direct usage (via the --plot option).

$ mdbenchmark analyze
                   gromacs  nodes  ns/day  run time [min]    gpu        host  ncores
0  gromacs/5.1.4-plumed2.3      1  10.878              15  False       draco      32
1  gromacs/5.1.4-plumed2.3      2   21.38              15  False       draco      64
2  gromacs/5.1.4-plumed2.3      3  34.033              15  False       draco      96
3  gromacs/5.1.4-plumed2.3      4  40.274              15  False       draco     128
4  gromacs/5.1.4-plumed2.3      5   51.71              15  False       draco     160

It is possible to define your own host templates in addition to the ones shipped with the package. A template file should have the same filename as the UNIX command hostname returns to be detected automatically. Otherwise you can point MDBenchmark to a specific template by providing its name via the --host option.

Assuming you created a new host template in your home directory ``~/.config/MDBenchmark/my_custom_hostfile~~:

$ mdbenchmark generate protein --host my_custom_hostfile --module gromacs/5.1.4-plumed2.3

Here is an example job template for the MPG cluster hydra.

# @ shell=/bin/bash
#
# @ error = {{ name }}.err.$(jobid)
# @ output = {{ name }}.out.$(jobid)
# @ job_type = parallel
# @ node_usage = not_shared
# @ node = {{ n_nodes }}
# @ tasks_per_node = 20
{%- if gpu %}
# @ requirements = (Feature=="gpu")
{%- endif %}
# @ resources = ConsumableCpus(1)
# @ network.MPI = sn_all,not_shared,us
# @ wall_clock_limit = {{ formatted_time }}
# @ queue

module purge
module load {{ module }}

# run {{ module }} for {{ time }} minutes
poe gmx_mpi mdrun -deffnm {{ name }} -maxh {{ time / 60 }}

MDBenchmark passes the following variables to each template:

To ensure correct termination of jobs formatted_time is 5 minutes longer than time.

MDBenchmark will look for user templates in the xdg config folders defined by the environment variables XDG_CONFIG_HOME and XDG_CONFIG_DIRS which by default are set to $HOME/.config/MDBenchmark and /etc/xdg/MDBenchmark, respectively. If the variable MDBENCHMARK_TEMPLATES is set, the script will also search in that directory.

MDBenchmark will first search in XDG_CONFIG_HOME and XDG_CONFIG_DIRS for a suitable template file. This means it is possible to overwrite system-wide installed templates or templates shipped with the package.

Contributions to the project are welcome! Information on how to contribute to the project can be found in CONTRIBUTING.md and DEVELOPER.rst.