Jumel (JUROPA memory logger)

General information

The JUROPA memory logger is intended for monitoring the memory usage of applications on NUMA architectures (especially JUROPA). It consists currently of two Python scripts:

• jumel (JUROPA memory logger), the actual logger
• juman (JUROPA memory analyzer), a postprocessing tool

Concept

The logger is started with mpiexec and subsequently starts the application to monitor. It will create a directory (default: .memlog in the PBS_O_WORKDIR) and by default each task will create its own logfile in that directory. After the run is finished the analyzer juman is run in the PBS_O_WORKDIR, either from within the same job script or afterwards on the login node to analyze the consumed resources.

Usage

In order to get an overview of the valid options please use

jumel -u
juman -u

Suppose the application to monitor is started usually as follows:

mpiexec -np 32 -e APP_ROOT app.x -i app.inp

To start the application with jumel use

mpiexec -np 32 -e PBS_ID,APP_ROOT jumel -a "app.x -i app.inp"

The variable PBS_ID needs not to be specified, how ever it will be displayed in the jumel logfiles and eases the tracking of the runs afterwards (e.g. when looking for the job in the system logfiles).

Once the run is finished run juman in the same directory:

juman -s all -i

The -s option switches on the statistics (currently: total consumption and maximum consumption per time step) and the -i option will start the graphical display of the results.

Currently only a gnuplot interface is implemented and postscript of xfig files can be generated. An interface for visualization with Python is planned.

Example: Namd

The following job script was used to monitor running the apoa1 benchmark with Namd:

#!/bin/bash 
#MSUB -l nodes=4:ppn=8
#MSUB -l walltime=00:15:00
#MSUB -v tpt=1 

module load namd/2.7

mpiexec -np 32 -e PBS_JOBID jumel -a "$NAMD_ROOT/bin/namd2 apoa1.namd"

The graphs were optained afterwards on the login node with

juman -s all -i

SVN Access

svn list https://svn.version.fz-juelich.de/jumel