Skip to content

BQCD

BQCD is a hybrid Monte-Carlo code that simulates Quantum Chromodynamics with dynamical standard Wilson fermions. The computations take place on a four-dimensional regular grid with periodic boundary conditions. The kernel of the program is a standard conjugate gradient solver with even/odd pre-conditioning. Several HPC systems were used to compare the hybrid communication mode of BQCD against the pure MPI.

Original version of the code is a combination of both “hybrid” MPI/OpenMP plus shmem communication. Some of the numerical kernels within BQCD are in the process of being ported to OpenCL.

Principal Investigator:

Dr David Brayford (LRZ) Email: brayford@lrz.de

Dr. Momme Allalen (LRZ) Email: allalen@lrz.de

Dr. Hinnerk Stueben (Konrad-Zuse-Zentrum fuer Informationstechnik Berlin) Email: stueben@zib.de

Other application users/developers:

QCD codes are widely used in many projects like LHC, QPACE, PRACE for research or benchmarking purposes. BQCD, written by Dr. Hinnerk Stueben (Konrad-Zuse-Zentrum für Informationstechnik Berlin) and improved by Dr. Momme Allalen (Leibniz Supercomputing Centre). It supports both pure MPI and hybrid programming model. In addition to being a well-used QCD application in Germany, it is also used in research centres or Universities in Japan, UK, Switzerland, Russia, Mexico and Yemen.

Scientific area:

Quantum Chromodynamics with dynamical standard Wilson fermions.

Scalability:

Up to the full BG/P 294912 cores. The scalability is also proven up to 131072 cores of SuperMUC (IBM iDATAPLEX cluster @ LRZ).

Versions:

  • MPI
  • OpenMP

Tested on platforms:

BG/L, BG/P, Cray XT4/XT5 and XT6, SGI Altix/UV, ICE Cluster, Tibidabo (ARM prototype system at BSC) and MinoTauro (GPU cluster at BSC).

URL: BQCD web page

References:

  1. G. Schierholz and H. Stüben, Optimizing the Hybrid Monte Carlo Algorithm on the Hitachi SR8000, in S. Wagner, W. Hanke, A. Bode and F. Durst, High Performance Computing in Science and Engineering, Munich 2004, Springer-Verlag, pp. 385–393.
  2. M. Allalen, M. Brehm and H. Stüben, Performance of quantum chromodynamics (QCD) simulations on the SGI Altix 4700, COMPUTATIONAL METHODS IN SCIENCE AND TECHNOLOGY 14(2), 69-75 (2008).
  3. http://www.zib.de/stueben/bqcd
  4. Stüben, H. and Allalen, M. Extreme Scaling of the BQCD Benchmark, in: Mohr, B. and Frings, W. (eds.), Technical Report FZJ-JSC-IB-2010-03 (2010) 31–34

BIGDFT

BigDFT is a DFT massively parallel electronic structure code (GPL license) using a wavelet basis set. Wavelets form a real space basis set distributed on an adaptive mesh (two levels of resolution in our implementation). GTH or HGH seudopotentials are used to remove the core electrons. Thanks to our Poisson solver based on a Green function formalism, periodic systems, surfaces and isolated systems can be simulated with the proper boundary conditions.

Principal Investigator:

INAC
CEA
Email: Thierry.Deutsch@cea.fr

Other application users/developers:

INAC (CEA-Grenoble), Basel University

Scientific area:

Computational Material Science, Quantum Chemistry

Scalability:

>16 384 IBM BG/Q cores The scalability of the code is also proven on up to 288 nVIDIA M2090 GPUs on CURIE (BULL cluster @ TGCC/GENCI, France)

Versions:

  • MPI
  • OpenMP
  • CUDA
  • OpenCL

Tested on platforms:

BG/P, BG/Q, Cray + GPU (Titan, Oak Ridge), Intel Sandy Bridge, ARM

URL: BigDFT website

COSMO

COSMO-Model is a non-hydrostatic limited-area atmospheric prediction model. It has been designed for both operational numerical weather prediction (NWP) and various scientific applications on the meso-scale. The COSMO-Model is based on the primitive thermo-hydrodynamical equations describing compressible flow in a moist atmosphere. The model equations are formulated in rotated geographical coordinates and a generalized terrain following height coordinate. A variety of physical processes are taken into account by parameterization schemes. Besides the forecast model itself, a number of additional components such as data assimilation, interpolation of boundary conditions from a driving model, and post-processing utilities are required to run the model in NWP-mode, climate mode or for case studies.

Principal Investigator:

COSMO European Consortium
Piero Lanucara, CINECA – ITALY
Email: p.lanucara@cineca.it

Other application users/developers:

Consortium for Small Scale Modeling includes meteorological services centers (Italian Air Force USAM/CNMCA, German DWD, Switzerland MeteoSwiss among the others) regional services and institutions. CINECA is involved in national and international agreement and HPC services within COSMO consortium.

Scientific area:

Weather and climate.

Scalability:

Strong scaling up to thousands cores on CRAY XC30 and/or similar computing architectures (COSMO/OPCODE)

Versions:

  • MPI
  • OpenMP
  • CUDA

Other:

OpenAcc

Tested on platforms:

COSMO is portable among all existing HPC platforms (including “heterogeneous” machines)

More information:

http://www.cosmo-model.org/
POMPA priority project: http://www.cosmo-model.org/content/tasks/priorityProjects/pompa/default.htm
HP2C COSMO-CCLM development on hpcforge: https://hpcforge.org/projects/cclm-dev/
METEO@CINECA http://www.cineca.it/content/meteo-e-clima

EUTERPE

EUTERPE is a particle-in-cell (PIC) gyrokinetic code for global linear and non-linear simulations of fusion plasma instabilities in three-dimensional geometries, in particular in tokamaks and stellarators. Simulations of plasma micro-instabilities and related turbulence are a necessary complement for stellarator and tokamak experiments such as Wendelstein 7-X and ITER. Especially important are full torus simulations for three-dimensional stellarator configurations. For this purpose the EUTERPE code has been developed which originally solved the electrostatic gyrokinetic equation globally in arbitrary three-dimensional geometry. The full kinetic treatment of the electrons and the inclusion of magnetic field perturbations extends the scope of applicability to the magnetohydrodynamic (MHD) regime and will make EUTERPE the first code worldwide that is able to simulate global electromagnetic instabilities in three dimensions.

Principal Investigator:

Ralf Kleiber
Max-Planck Institut für Plasmaphysik
Email: ralf.kleiber@ipp.mpg.de

Roman Hatzky
Max-Planck Institut für Plasmaphysik
Email: roman.hatzky@ipp.mpg.de

Other application users/developers:

Laboratorio Nacional de Fusión (CIEMAT), Madrid, Spain Barcelona Supercomputing Center (BSC), Barcelona, Spain Centre de Recherches en Physique des Plasmas (EPFL-PPB), Lausanne, Switzerland.

The Euterpe code should be obtained freely under the approval of their authors (Contact: Ralf Kleiber, Roman Hatzky)

Scientific area:

Plasma Physics

Scalability:

EUTERPE has performed well for up to 61440 cores on the Jugene supercomputer (BG/P)

Versions:

MPI
BSC has developed a hybrid version (MPI+OpenMP)

Tested on platforms:

PowerPC, Intel, BG/P, Cell, Cray

References:

G. Jost, T. M. Tran, W. Cooper, and K. Appert. Phys. Plasmas 8: 3321 (2001)
V. Kornilov, R. Kleiber, R. Hatzky, L. Villard, and G. Jost. Phys. Plasmas 11: 3196 (2004)
V. Kornilov, R. Kleiber, and R. Hatzky, Nucl. Fusion 45: 238 (2005)
R. Kleiber, Global linear gyrokinetic simulations for stellarator and axisymmetric equilibria, Joint Varenna-Lausanne International Workshop. AIP Conference Proceedings, 871, p. 136, 2006
E. Sánchez, R. Kleiber, R. Hatzky, et al., IEEE Transaction on Plasma Science 38, 2119 (2010).
R. Kleiber, R. Hatzky, and A. Mishchenko, Contributions to Plasma Physics 50, 766 (2010).
R. Kleiber, R. Hatzky, A. Könies, K. Kauffmann, and P. Helander, Computer Physics Communications 182, 1005 (2011).

 

MP2C

MP2C implements the multi-particle collision dynamics method, which is a particle based description of hydrodynamics taking into account thermal fluctuations and making it possible to simulate flow phenomena on a mesoscopic level. In addition, a molecular dynamics part allows the simulation of particles, e.g. colloids or polymers, which are coupled to the hydrodynamics description in a consistent way on the particle level, which allows to take into account a hydrodynamic coupling between solvated particles, relevant for structural and dynamic effects. Different boundary conditions allow various experimental settings, like shear boundary conditions or channel flow. To account for an inhomogeneous distribution of solute particles, a load-balancing strategy, based on an unstructured sub-domain distribution, is implemented to support parallel scaling of the program.

Principal Investigator:

Godehard Sutmann
Forschungszentrum Jülich, JSC
Email: g.sutmann@fz-juelich.de

Rene Halver
Forschungszentrum Jülich, JSC
Email: r.halver@fz-juelich.de

Other application users/developers:

Forschungszentrum Jülich institutes (ICS, PGI, IEK)

Scientific area:

Soft Matter Physics

Scalability:

458,752 cores on BlueGene/Q 262,144 cores on BlueGene/P 17,664 cores (JUROPA cluster)

Versions:

MPI

OmpSs(+OpenCL) Version under development for the project

Tested on platforms:

BlueGene/Q, /P and /L
x86 (JUROPA cluster / JUDGE cluster)
CRAY XT4/XT5

PEPC

PEPC is a tree code for solving the N-body problem. It is not restricted to Coulomb systems but also handles gravitation and hydrodynamics using the vortex method as well as smooth particle hydrodynamics (SPH). PEPC is a non-recursive version of the Barnes-Hut algorithm with a level-by-level approach to both tree construction and traversals. The parallel version is a hybrid MPI/PThreads implementation of the Warren-Salmon ‘Hashed Oct-Tree’ scheme. The long range interactions are computed using multipole groupings of distant particles to reduce computational effort. The Barnes-Hut algorithm is well suited to dynamic, nonlinear problems and can be combined with multiple-timestep integrators.

Principal Investigator:

P. Gibbon, Simulation Laboratory Plasma Physics, Jülich Supercomputing Centre
Email: pepc@fz-juelich.de

Other application users/developers:

Imperial College London, University of Illinois, FZ Jülich, MPIfR Bonn, TU Chemnitz, Institute of Physics ASCR Prague and many more.

Scientific area:

N-body simulations (Coulomb, Gravitational, …) beam-plasma interaction, vortex dynamics, gravitational interaction, MD

Scalability:

294912 cores on BlueGene/P 458752 cores on BlueGene/Q The code has been qualified for being part of the High-Q Club which gathers the highest scaling codes on JUQUEEN (IBM BG/Q @ JSC, Germany) with a scaling on the full system i.e 458,752 cores (1,668,196 parallel threads) on BlueGene/Q

Versions:

MPI
pthreads, OmpSs

Tested on platforms:

BlueGene/P, BlueGene/Q, x86, ARM

Portable to all existing HPC platforms

URL: www.fz-juelich.de/ias/jsc/pepc/

[1] M. Winkel, R. Speck, H. Hübner, L. Arnold, R. Krause, P. Gibbon
“A massively parallel, multi-disciplinary Barnes-Hut tree code for
extreme-scale N-body simulations”
Computer Physics Communications, Vol. 183 (4), April 2012

PROFASI

PROFASI is a Monte Carlo simulation package for protein folding and aggregation simulations written in C++ (now, partially C++11). It implements an all-atom protein model, an implicit solvent interaction potential and several modern Monte Carlo methods for simulation of systems with rough energy landscapes. It has been used to study folding and thermodynamics of helical, beta sheet and mixed proteins of up to 92 amino acids. It has also been used to study the aggregation of disease related peptides from 6 to 140 amino acids and the thermal/mechanical unfolding of many proteins.

As of April 2013, Top7 is the largest and most complex protein folded with an all-atom protein model using unconstrained physical simulations from random initial conformations. This simulation was done with PROFASI.  (http://onlinelibrary.wiley.com/doi/10.1002/prot.24295/pdf)

Principal Investigator:

Sandipan Mohanty, Jülich Supercomputing Centre, Email: s.mohanty@fz-juelich.de

Other application users/developers:

Main collaborator in PROFASI development:
Anders Irbäck, Lund University, Sweden

Scientific area:

Computational Biophysics

Scalability:

Weak scaling to 16384 cores on BGP and 4096 cores on x86_64 based clusters.

Versions:

MPI

C++11 threads / pthreads

Tested on platforms:

x86/ARM based clusters, BG/P, BG/Q

URL: http://zam581.zam.kfa-juelich.de/jsc/slbio/PROFASI_Gallery/

QUANTUM ESPRESSO

Quantum ESPRESSO is an integrated suite of Open-Source computer codes for electronic-structure calculations and materials modeling at the nanoscale. It is based on density-functional theory, plane waves, and pseudopotentials. Quantum ESPRESSO has evolved into a distribution of independent and inter-operable codes in the spirit of an open-source project.

The QuantumESPRESSO distribution consists of a “historical” core set of packages and a set of plug-ins that perform more advanced tasks. Researchers active in the field of electronic-structure calculations are encouraged to participate in the project by contributing their own codes or by implementing their own ideas into existing codes. Quantum ESPRESSO is an initiative of the DEMOCRITOS National Simulation Center (Trieste) and SISSA (Trieste), in collaboration with CINECA National Supercomputing Center, the Ecole Polytechnique Fédérale de Lausanne, Université Pierre et Marie Curie, Princeton University, and Oxford University.

Courses on modern electronic-structure theory with hands-on tutorials on the Quantum ESPRESSO codes are offered on a regular basis in collaboration with the Abdus Salam International Centre for Theoretical Physics in Trieste.

Principal Investigator: 

Nico Sanna, CINECA – Roma, ITALY
Email: n.sanna@cineca.it

Other application users/developers: 

DEMOCRITOS  National Simulation Center (Trieste)
SISSA  (Trieste)
CINECA  National Supercomputing Center (Bologna)
Ecole Polytechnique Fédérale de Lausanne
Université Pierre et Marie Curie (Paris)
Princeton University
Oxford University

Scientific area: 

Material Science, Computational Quantum Chemistry

Scalability: 

Strong scaling up to ten thousands cores on IBM Blue-Gene Q and/or similar computing architectures. The code has been scaled out to up to 32 768 physical cores (and 65 536 virtual cores) of a IBM BG/Q system (Fermi @ CINECA, Italy)

Versions: 

MPI
OpenMP
CUDA

Tested on platforms: 

Quantum ESPRESSO is portable among all existing HPC platforms (including “heterogeneous” machines)

URL: www.quantum-espresso.org/

SMMP

SMMP (Simple Molecular Mechanics for Proteins) provides advanced Monte Carlo algorithms and several force fields to simulate the thermodynamics of single proteins and assemblies of peptides. PySMMP, SMMP’s Python bindings, allow for rapid prototyping of new algorithms and provide a convenient way to implement complex work flows.

Principal Investigator:

Jan H. Meinke, JSC, Email: j.meinke@fz-juelich.de

Other application users/developers:

SMMP is used by scientific groups all over the world. The main developers are Ulrich Hansmann at the University of Oklahoma, Shura Haryan and C.K. Hu at the Academia Sinica, Frank Eisenmenger at the Leibniz Institute for Molecular Pharmacology, and Jan Meinke and Sandipan Mohanty at the Jülich Supercomputing Centre.

Scientific area:

Protein folding and protein aggregation

Scalability:

32768 core on a BG/P. 4096 cores on JuRoPA

Versions:

  • MPI
  • OpenMP
  • CUDA
  • OpenCL

Tested on platforms:

SMMP has been tested on a large variety of platforms including IBM BG/Q, IBM Cell BE, Intel Xeon Phi, and x86 . It should compile and run on any platform with a Fortran 90 compiler and (optionally) MPI.

URL: http://smmp.berlios.de

SPECFEM3D

SPECFEM3D uses the continuous Galerkin spectral-element method to simulate acoustic (fluid), elastic (solid), coupled acoustic/elastic, poroelastic or seismic wave propagation in any type of conforming mesh of hexahedra (structured or not). It can for instance model seismic waves propagating in sedimentary basins or any other regional geological model following earthquakes; the whole Earth can also be simulated. It can also be used for non destructive testing or for ocean acoustics.

Principal investigator:

Dimitri Komatitsch, CNRS Marseille, France,
Email: komatitsch@lma.cnrs-mrs.fr

• Winner of the Gordon Bell award, 2003
• Finalist of the Gordon Bell award, 2008
• Winner of the Bull-Joseph Fourier award, 2010.

Jeroen Tromp, Princeton Univ, USA
Email: jtromp@princeton.edu

Other application users/developers:

It is used by more than 300 sites in academia and industry. See the latest download map below, as of April 2013.

Scientific area:

Acoustic, elastic, viscoelastic, poroelastic wave propagation: seismic waves, earthquakes, non-destructive testing, ocean acoustics.

Scalability:

Up to 693,600 cores on IBM BlueWaters, sustained petaflop/s reached in February 2013.

Versions:

MPI
CUDA

Tested on platforms:

SPECFEM3D is portable among all existing HPC platforms.

SPECFEM3D has been completely ported into OpenCL on nVIDIA GPUs with the same performance as the CUDA version.

URL: http://www.geodynamics.org/cig/software

More images and videos:

See http://komatitsch.free.fr  and  http://global.shakemovie.princeton.edu

YALES2

YALES2 is a research code that aims at the solving of two-phase combustion from primary atomization to pollutant prediction on massive complex meshes. The code is used and developed by more than 100 researchers at CORIA and in several laboratories that are grouped in a joint initiative called SUCCESS in order to promote super-computing and help in the training and the porting of the AVBP and YALES2 codes. YALES2 is also used in the aeronautical, automotive and process engineering industries. YALES2 is able to handle efficiently unstructured meshes with several billions of elements, thus enabling the DNS and LES of laboratory and semi-industrial configurations. The solvers of YALES2 cover a wide range of phenomena and applications and they may be assembled to address multi-physics problems.

Principal Investigator:

V. Moureau, CORIA,
Email: vincent.moureau@coria.fr
• 3rd of the Bull-Joseph Fourier prize in 2009
• 2011 IBM faculty award

G. Lartigue, CORIA
Email: ghislain.lartigue@coria.fr

Other application users/developers:

It is used by more than 60 people in labs and in the industry:

Labs : CORIA, I3M, LEGI, EM2C, IMFT, CERFACS, IFP-EN, ULB, …

Industry : Safran, Airbus, AirLiquide, Rhodia, Renault, Areva, GDF Suez, …

Development discontinued

Scientific area:

Computational Fluid Dynamics, combustion

Scalability:

YALES2 solves the low-Mach number Navier-Stokes equations with a projection method for constant and variable density flows. These equations are discretized with a 4th-order central scheme in space and a 4th-order Runge-Kutta like scheme in time. The efficiency of projection approaches is usually driven by the performances of the Poisson solver. In YALES2, the linear solver is a highly efficient Deflated Preconditioned Conjugate Gradient that has two mesh levels. As a result, YALES2 is currently used for production runs with meshes of 18 billion cells on 16384 cores of the Curie machine, up to 16384 cores on IBM BG/P and BULL x86 systems.

Versions:

MPI
OpenMP

Tested on platforms:

YALES2 has been ported on the major HPC plateforms: Intel clusters (Curie and Arain at CEA, Antares at CRIHAN), Blue Gene machines (P and Q at IDRIS and JUELICH), ARM cluster and Intel Xeon Phi.

URL: http://www.coria-cfd.fr/index.php/Main_Page