Version 18 (modified by 7 years ago) ( diff ) | ,
---|
OpenFOAM - Build Instructions for JURECA
OpenFOAM is free, open-source software for computational fluid dynamics.
- OpenFOAM Foundation
- OpenCFD Ltd - owner of the trademark OpenFOAM
OpenFOAM releases are scheduled every six months in June and December. Releases designated OpenFOAM+ contain client-sponsored developments of which much has been transferred to, but not released in the OpenFOAM Foundation branch.
Preliminaries
- General build instructions can be found here:
- There are, roughly, two components to OpenFOAM:
- the actual CFD code FOAM -- definitely to be compiled with ICC
- the GUI analysis tools, based on ParaView (using the Qt GUI toolkit)
- All OpenFOAM applications do parallel communication through the Pstream library only.
So using any underlying message passing library (MPI, etc.) is just a matter of replacing Pstream. more...
- Some of the ThirdParty packages need to be built first. These third party sources are required, but are not bundled. The source tarballs could be expanded in the ThirdParty-${VERSION} directory, but we use installed modules instead.
- Boost - 1.61.0 works
- CGAL - 4.8.2 works
- GMP - 6.1.2 works
- MPFR - 3.1.5 works
- METIS - 5.1.0 is expected METIS Official Website
### set env. variables we will use VERSION=3.0.1 INST_DIR=/homeb/zam/${USER}/my_OpenFOAM mkdir $INST_DIR cd $INST_DIR
Download & Extract
Source packages are available from the download section of the OpenFOAM website:
The source code must be unpacked into the top-level directory of the final installation.
### download OpenFOAM wget https://downloads.sourceforge.net/project/foam/foam/${VERSION}/OpenFOAM-${VERSION}.tgz wget https://downloads.sourceforge.net/project/foam/foam/${VERSION}/ThirdParty-${VERSION}.tgz # extract OpenFOAM tar -xzf OpenFOAM-${VERSION}.tgz -C $INST_DIR tar -xzf ThirdParty-${VERSION}.tgz -C $INST_DIR
Setup OpenFOAM
Patch
Patch $INST_DIR/OpenFOAM-${VERSION}/etc/bashrc
### bashrc + foamInstall=${INST_DIR} + : ${WM_COMPILER:=Gcc}; export WM_COMPILER + : ${WM_MPLIB:=SYSTEMOPENMPI}; export WM_MPLIB
- Instead of setting variable in bashrc, they can be set via environment variables before you call bashrc later.
export FOAM_INST_DIR=...
overwrites foamInstallexport WM_COMPILER=...
, default is Gcc, should be set to Iccexport WM_MPLIB=...
, default is SYSTEMOPENMPI, should be set to EASYBUILDMPI
Patch $INST_DIR/OpenFOAM-${VERSION}/etc/config/settings.sh
### settings.sh Linux) ... x86_64) ... 64) WM_ARCH=linux64 export WM_COMPILER_LIB_ARCH=64 + export WM_CC=${CC} # for use in non-wmake (e.g. ThirdParty) compilations + export WM_CXX=${CXX} # for use in non-wmake (e.g. ThirdParty) compilations + export WM_CFLAGS="${CFLAGS} -fPIC" # for use in non-wmake (e.g. ThirdParty) compilations + export WM_CXXFLAGS="${CXXFLAGS} -fPIC" # for use in non-wmake (e.g. ThirdParty) compilations + export WM_LDFLAGS= # for use in non-wmake (e.g. ThirdParty) compilations
### settings.sh + #unset MPI_ARCH_PATH MPI_HOME FOAM_MPI_LIBBIN + EASYBUILDMPI) + export FOAM_MPI=mpi + ;;
Patch $INST_DIR/OpenFOAM-${VERSION}/wmake/rules
### linux64Gcc/c, linux64Icc/c + cc = ${MPICC} ### linux64Gcc/c++, linux64Icc/c++ + CC = ${MPICXX} -std=c++0x -fp-trap=common -fp-model precise ### linux64Gcc/cOpt, linux64Icc/cOpt + cOPT = ${CFLAGS} ### linux64Gcc/c++Opt, linux64Icc/c++Opt + c++OPT = ${CXXFLAGS}
- examples: https://github.com/OpenFOAM/OpenFOAM-dev/tree/master/wmake/rules
- MPI: ${MPICC} required to compile finiteVolume/lnInclude/fvPatchField.C
Set Environment
The environment must be set up according to where the source code directories are located.
INST_DIR=/homeb/zam/${USER}/my_OpenFOAM # must not end with a '/', or build will fail with "touch [...] permission denied" cd $INST_DIR # load modules module --force purge module load Stages/Devel module load Intel/2017.0.098-GCC-5.4.0 module load IntelMPI/2017.0.098 module load flex/2.5.39 # flex 2.6.0 causes errors module load Boost/1.61.0-Intel-2017.0.098 module load MPFR/3.1.5-Intel-2017.0.098 module load GMP/6.1.2-Intel-2017.0.098 module load CGAL/4.8.2-Intel-2017.0.098 module load METIS/5.1.0-Intel-2017.0.098 # configure environment export FOAM_INST_DIR=$INST_DIR export WM_COMPILER=Icc export WM_MPLIB=EASYBUILDMPI export MPICC=mpicc export MPICXX=mpiicpc export CC=icc export CXX=icpc export CFLAGS="-no-prec-div -O3 -xHost -fp-model precise -diag-disable 654,1125" export CXXFLAGS="-O3 -xHost -fp-model precise -diag-disable 654,1125 -DMPICH_SKIP_MPICXX" export WM_NCOMPPROCS=8 # parallel build export WM_LABEL_SIZE=32 # 32bit for labels/indices - relevant for METIS (if 64 needed patch METIS first) # load environment VERSION=3.0.1 source OpenFOAM-${VERSION}/etc/bashrc foam foamSystemCheck
- build with Intel 2017.1.138: OpenFOAM/config_Intel-2017.1.138
- build with Stage/2016a OpenFOAM/config_2016a
- build with Stage/2016a + Score-P OpenFOAM/config_2016a_ScoreP
- build for Intel/Xeon Phi (Knights Landing) OpenFOAM/config_knl
Setup ThirdParty
OpenFOAM has configs which set where it expects to find its "third party" packages.
There are in $FOAM_ETC/config/
as various shell scripts.
The top-level ./Allwmake will build Scotch, but not CGAL, Boost, GMP, and MPFR.
Those packages need to be built separately, and installed into the appropriate directory.
CGAL
Patch $INST_DIR/OpenFOAM-${VERSION}/etc/config/CGAL.sh
### CGAL.sh + boost_version=$EBVERSIONBOOST + cgal_version=$EBVERSIONCGAL + export BOOST_ARCH_PATH=$EBROOTBOOST + export CGAL_ARCH_PATH=$EBROOTCGAL
- Boost -installed as module Boost/1.61.0-Intel-2017.0.098
- GMP - installed as module GMP/6.1.2-Intel-2017.0.098
- MPFR - installed as module MPFR/3.1.5-Intel-2017.0.098
METIS
Patch $INST_DIR/OpenFOAM-${VERSION}/etc/config/metis.sh
# metis.sh + export METIS_ARCH_PATH=$EBROOTMETIS
Patch $WM_THIRD_PARTY_DIR/Allwmake
# $WM_THIRD_PARTY_DIR/Allwmake + if [ -d "$METIS_ARCH_PATH" ] # if [ -d "$METIS_VERSION" ] + then + if [ -f $METIS_ARCH_PATH/include/metis.h \ + -a -r $METIS_ARCH_PATH/lib/libmetis.so ] + then + cp $METIS_ARCH_PATH/lib/libmetis.so $FOAM_EXT_LIBBIN + echo " metis header in $METIS_ARCH_PATH/include" + echo " metis libs in $FOAM_EXT_LIBBIN"
- ATTENTION: metis.h
- METIS library's IDXTYPEWIDTH must fit to $WM_LABEL_SIZE (for METIS/5.1.0-Intel-2017.0.098: IDXTYPEWIDTH=32)
- METIS library's REALTYPEWIDTH must fit to $WM_PRECISION_OPTION (for METIS/5.1.0-Intel-2017.0.098: REALTYPEWIDTH=64)
Scotch
Scotch decomposition which requires no geometric input from the user and attempts to minimise the number of processor boundaries.
Patch $WM_THIRD_PARTY_DIR/etc/wmakeFiles/scotch/Makefile.inc.i686_pc_linux2.shlib-OpenFOAM-64Int32
# scotch/Makefile.inc.i686_pc_linux2.shlib-OpenFOAM-64Int32 + AR = ${CC} + CCS = ${CC} + CCP = ${MPICC} + CCD = ${MPICC} + CFLAGS = -O3 -xHost -fp-model precise -DCOMMON_FILE_COMPRESS_GZ -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict=__restrict
Copy source to required location.
source $INST_DIR/OpenFOAM-${VERSION}/etc/config/scotch.sh mkdir -p $SCOTCH_ARCH_PATH cp -a $WM_THIRD_PARTY_DIR/scotch_6.0.3/* $SCOTCH_ARCH_PATH
Build & Test OpenFOAM
build OpenFOAM
# cd $WM_PROJECT_DIR foam # clean if this is a rebuild wclean all # build wmake ./wmake/makeWmake # build OpenFOAM (except Qt & ParaView) ./Allwmake
test OpenFOAM
Validate the build by running
# cd $WM_PROJECT_DIR foam foamInstallationTest # create the user run directory mkdir -p $FOAM_RUN # test with a simple tutorial: run cp -r $FOAM_TUTORIALS/incompressible/simpleFoam/pitzDaily ./ cd pitzDaily blockMesh simpleFoam # test with a more advanced tutorial # details here: https://cfd.direct/openfoam/user-guide/cavity#x5-40002.1 run cp -r $FOAM_TUTORIALS/incompressible/icoFoam/cavity ./ cd cavity foamRunTutorials #paraFoam
- More detailed tutorials to test can be found here:
Build Tools
ParaView
OpenFOAM is shipped with ParaView for post-processing OpenFOAM field results.
This requires a separate compilation which can be omitted if using other post-processing software.
cd $WM_THIRD_PARTY_DIR ./makeParaView
References
- flex 2.6.0
- 2.6.0 causes errors: undefined reference to `yyFlexLexer::yywrap()'
- https://www.cfd-online.com/Forums/openfoam-installation/163986-building-openfoam-3-0-archlinux.html
- -fp-model precise
- Disables optimizations that are not value-safe on floating-point data.
- https://www.nccs.nasa.gov/images/FloatingPoint_consistency.pdf
- -DMPICH_SKIP_MPICXX
- -diag-disable 654
- hide warning #654:
- -diag-disable 1125
- hide warning #1125: function "..." is hidden by "..." -- virtual function override intended?