This release contains several bug fixes for version 3.

Features

• Make scaling projection show optimal scaling. (#193)

Bugfixes

• Pin markupsafe to 2.0.1 to fix incompatibility with jinja2 ^2.10. (#184)
• Fix bug when plotting cpu-/gpu-data only with 'version3-categories'. (#192)
• Don't use deprecated kwarg 'type' in matplotlib.pyplot.savefig. (#195)

This is a new major release that changes the inner workings of MDBenchmark and prepares it for more modularity. Most importantly, users can now scan the nubmer of MPI ranks and OpenMP threads with GROMACS.

Features

• Add ability to scan number of MPI ranks and OpenMP threads. (#165)
• Add support to run multiple simulations on single nodes (GROMACS-only). (#168)

3.0.0

This version was skipped intentionally. Version 3.0.1 is the first release of the major version 3.

Literature

Please cite the latest MDBenchmark publication if you use the tool to benchmark your simulations. This will help raise awareness of benchmarking and help people improve their simulation performance, as well as reduce overall resource wastage.

M. Gecht, M. Siggel, M. Linke, G. Hummer, J. Köfinger MDBenchmark: A toolkit to optimize the performance of molecular dynamics simulations. J. Chem. Phys. 153, 144105 (2020); https://doi.org/10.1063/5.0019045

This release contains several bug fixes around long standing issues. In addition, the startup time of the command-line interface was lowered. This should be immediately noticeable upon usage.

Changelog

Features

• Add GPUs to cobra template. (#142)
• Lower startup time of the CLI. (#153)

Bugfixes

• Already submitted benchmarks are now hidden in the summary of mdbenchmark submit. (#139)
• Plotting will now work, even when some benchmarks are unfinished. (#140)
• Restarting benchmarks with mdbenchmark submit --force works again. (#141)

Misc

• Use poetry to manage Python dependencies and the project in general. (#155)

After lots of work, MDBenchmark 2.0 is finished. This new release includes major improvements to the plotting functionality and user experience in general. But it also comes with breaking changes to the command-line interface and requires an update to your Python environment.

If you have any workflows that rely on the function of specific calls to mdbenchmark, please check the latest CLI reference to make sure everything works as expected.

Highlights

• Revamped plotting functionality, making it possible to plot many different benchmark combinations (modules, hosts, other settings) at once.
  • The old mdbenchmark analyze --plot command was replaced with mdbenchmark plot.
  • Plots can now be easily customized with the --dpi, --font-size, --xtick-step, --no-fit and --no-watermark options.
• All calls to the main mdbenchmark commands (generate, analyze, submit and plot) now use consistent option names, when handling similar functions.
• Running mdbenchmark analyze will not produce CSV output files, until asked to with the --save-csv option.
• mdbenchmark generate and mdbenchmark submit will now ask for confirmation, before performing any operations. The confirmation can be skipped with the --yes option.
• CPU and GPU benchmarks can now be generated at once, without the need to run two separate commands: mdbenchmark generate --cpu --gpu.
• The job name for the queuing system can be customized with the --job-name option when using mdbenchmark generate.

Installation

Please refer to our documentation on how to install the package from your favorite package manager.

Migrating

When upgrading from a previous version, you need to remove three packages that were previously installed alongside of MDBenchmark: datreant.core, datreant.data and mdsynthesis. Refer to our upgrading guide, if you need assistance.

Upgrading

To upgrade, use the appropriate command for your Python package manager.

conda

If you are using conda and your conda environment is called benchmark, simply run:

conda update -n benchmark mdbenchmark

pip

Users of pip run the following:

pip install mdbenchmark --upgrade

Full changelog

A full list of changes, both user facing and behind the scenes, can be found below.

Features

• Updated plotting interface. mdbenchmark analyze --plot is not deprecated. Use mdbenchmark plot instead.

  The new workflow for plotting data is as follows:

  1. Use mdbenchmark analyze --save-csv results.csv to generate a CSV output file.
  2. Use mdbenchmark plot --csv results.csv to plot the data.

  Consult mdbenchmark <command> --help for options to filter your data accordingly. (#52)

• mdbenchmark generate now accepts --cpu / --no-cpu and --gpu / --no-gpu. The default is --cpu and --no-gpu. (#69)

• Added user prompts to mdbenchmark generate and mdbenchmark submit. (#90)

• Added --yes flag to mdbenchmark generate and mdbenchmark submit to bypass user prompt. (#90)

• Added -nc and -ng options to mdbenchmark generate. These are short hand for --no-cpu and --no-gpu, respectively. (#93)

• Added template for MPCDF cluster cobra. (#104)

• Added --template and -t option to mdbenchmark generate, to specify a job template. The --host option still works. (#106)

• Standarize the CLI options across all mdbenchmark calls. (#107)

• Added mdbenchmark plot --dpi option to change the plot DPI. (#108)

• Added mdbenchmark plot --font-size to change the plot font size. (#108)

• Linear scaling fit can now be hidden with --no-fit. (#108)

• Updated ylim, xtick and ytick defaults. The steps for xtick can be overwritten with mdbenchmark plot --xtick-step. (#108)

• Added a watermark in the top left corner for every plot. Can be easiliy disabled with mdbenchmark plot --no-watermark. (#108)

• mdbenchmark analyze no longer writes CSV files by default. --save-csv flag added to write csv files. (#119)

• Added mdbenchmark generate --job-name to change the job name submitted to the queuing system. (#125)

Bugfixes

• Fixed a bug where benchmark creation with files ending in .namd did not work. (#124)
• Fixed a bug where benchmark creation would fail when the input file was not in the current directory. (#124)

Misc

• Replaced mdsynthesis with datreant and upgraded to the new datreant>=1.0 format. (#110)

This is the last release before we introduce a new command-line interface for mdbenchmark plot and some changes for mdbenchmark generate with version 2.0.0.

Bugfixes

• Users were unable to generate plots when running mdbenchmark analyze --plot. (#86)

Note

• This release was re-tagged to a newer commit on 2018-10-11 (11th October 2018), so the newly merged documentation is included in this tag.
• On 2018-10-15 (15th October 2018), we renamed this tag to a new style, from version-1.3.3 to v1.3.3.

This is probably the last release before we introduce a new command-line interface for mdbenchmark plot and some changes for mdbenchmark generate with version 2.0.0.

Bugfixes

• We now print all rows when running mdbenchmark analyze. (#68)
• Suppress UserWarning caused by MDAnalysis==0.18. (#71)

Misc

• Added new error message when running mdbenchmark generate [...] --skip-validation without providing a supported MD engine. (#74)

Bugfixes

• Module name validation is now performed case insensitive. (#61)

Misc

• Consolidate common functions from mdengines.gromacs and mdengines.namd into mdengines.utils, removing code duplication. (#57)
• Refactor unit tests. Make everything more concise and use some more pytest functionality. (#58)

Features

• Add functionality to perform benchmarks with NAMD. (#29)
• Consolidated internal API to output messages to the console. (#42)
• Module name is now validated against available modules on host. Can be
  skipped with --skip-validation. (#49)

Bugfixes

• The option --min-nodes needs to be bigger than --max-nodes. (#46)
• Fixed edge-case in input filename parsing. (#54)

Misc

• Fixed display of the number of benchmarks to-be generated. (#46)

Features

• GROMACS .tpr files can now be referenced with and without the file extension. (#32)
• Added Makefile to project. (#36)

Bugfixes

• fixed crash during analyze, if some simulations have not started/finished yet. (#26)
• Suppress FutureWarning caused by h5py. (#35)

Misc

• Updated docstrings and console output of generate, submit and analyze functions. (#36)
• Added more unit tests. (#36)