!!! GENERAL INSTRUCTION - NOT OPTIMIZED FOR PRODUCTION !!!
For production runs please check the installed OpenFOAM on JURECA with
module spider OpenFOAM
OpenFOAM - Build Instructions for JURECA
OpenFOAM is free, open-source software for computational fluid dynamics.
- OpenFOAM Foundation
- OpenCFD Ltd - owner of the trademark OpenFOAM
OpenFOAM releases are scheduled every six months in June and December. Releases designated OpenFOAM+ contain client-sponsored developments of which much has been transferred to, but not released in the OpenFOAM Foundation branch.
1. Preliminaries
- General build instructions can be found here:
- There are, roughly, two components to OpenFOAM:
- the actual CFD code FOAM -- definitely to be compiled with ICC
- the GUI analysis tools, based on ParaView (using the Qt GUI toolkit)
- All OpenFOAM applications do parallel communication through the Pstream library only.
So using any underlying message passing library (MPI, etc.) is just a matter of replacing Pstream. more...
- Some of the ThirdParty packages need to be built first. These third party sources are required, but are not bundled. The source tarballs could be expanded in the ThirdParty-${VERSION} directory, but we use installed modules instead.
- Boost - 1.61.0 works
- CGAL - 4.8.2 works
- GMP - 6.1.2 works
- MPFR - 3.1.5 works
- METIS - 5.1.0 is expected METIS Official Website
### set env. variables we will use
VERSION=3.0.1
# $INST_DIR must NOT (!!) end with a '/', or build will fail with "touch [...] permission denied"
INST_DIR=<my_openfoam_install_dir> # e.g. ${HOME}/OpenFOAM_install
mkdir -p $INST_DIR
2. Download & Extract
Source packages are available from the download section of the OpenFOAM website:
The source code must be unpacked into the top-level directory of the final installation.
For this HowTo this is $INST_DIR.
### download OpenFOAM
cd $INST_DIR
wget https://downloads.sourceforge.net/project/foam/foam/${VERSION}/OpenFOAM-${VERSION}.tgz
wget https://downloads.sourceforge.net/project/foam/foam/${VERSION}/ThirdParty-${VERSION}.tgz
# extract OpenFOAM
tar -xzf OpenFOAM-${VERSION}.tgz -C $INST_DIR
tar -xzf ThirdParty-${VERSION}.tgz -C $INST_DIR
3. Setup OpenFOAM
Patch
Patch $INST_DIR/OpenFOAM-${VERSION}/etc/bashrc
### bashrc
+ foamInstall=${INST_DIR}
+ : ${WM_COMPILER:=Gcc}; export WM_COMPILER
+ : ${WM_MPLIB:=SYSTEMOPENMPI}; export WM_MPLIB
- Instead of setting variable in bashrc, they can be set via environment variables before you call bashrc later.
export FOAM_INST_DIR=...overwrites foamInstallexport WM_COMPILER=..., default is Gcc, should be set to Iccexport WM_MPLIB=..., default is SYSTEMOPENMPI, should be set to EASYBUILDMPI
Patch $INST_DIR/OpenFOAM-${VERSION}/etc/config/settings.sh
### settings.sh
Linux)
...
x86_64)
...
64)
WM_ARCH=linux64
export WM_COMPILER_LIB_ARCH=64
+ export WM_CC=${CC} # for use in non-wmake (e.g. ThirdParty) compilations
+ export WM_CXX=${CXX} # for use in non-wmake (e.g. ThirdParty) compilations
+ export WM_CFLAGS="${CFLAGS} -fPIC" # for use in non-wmake (e.g. ThirdParty) compilations
+ export WM_CXXFLAGS="${CXXFLAGS} -fPIC" # for use in non-wmake (e.g. ThirdParty) compilations
+ export WM_LDFLAGS= # for use in non-wmake (e.g. ThirdParty) compilations
### settings.sh + #unset MPI_ARCH_PATH MPI_HOME FOAM_MPI_LIBBIN + EASYBUILDMPI) + export FOAM_MPI=mpi + ;;
Patch $INST_DIR/OpenFOAM-${VERSION}/wmake/rules
### linux64Gcc/c, linux64Icc/c
+ cc = ${MPICC}
### linux64Gcc/c++
+ CC = ${MPICXX} -std=c++0x
### linux64Icc/c++
+ CC = ${MPICXX} -std=c++0x -fp-trap=common -fp-model precise
### linux64Gcc/cOpt, linux64Icc/cOpt
+ cOPT = ${CFLAGS}
### linux64Gcc/c++Opt, linux64Icc/c++Opt
+ c++OPT = ${CXXFLAGS}
- examples: https://github.com/OpenFOAM/OpenFOAM-dev/tree/master/wmake/rules
- MPI: ${MPICC} required to compile finiteVolume/lnInclude/fvPatchField.C
Set Environment
The environment must be set up according to where the source code directories are located.
# set environment vars, if not set yet
# $INST_DIR must NOT (!!) end with a '/', or build will fail with "touch [...] permission denied"
#cd <my_openfoam_install_dir>
#export INST_DIR=$(pwd)
#export VERSION=3.0.1
cd $INST_DIR
# load modules
module --force purge
module use /usr/local/software/jureca/OtherStages
module load Stages/Devel
module load Intel/2017.0.098-GCC-5.4.0
module load IntelMPI/2017.0.098
module load flex/2.5.39 # flex 2.6.0 causes errors
module load Boost/1.61.0-Intel-2017.0.098
module load MPFR/3.1.5-Intel-2017.0.098
module load GMP/6.1.2-Intel-2017.0.098
module load CGAL/4.8.2-Intel-2017.0.098
module load METIS/5.1.0-Intel-2017.0.098
# configure environment
export FOAM_INST_DIR=$INST_DIR
export WM_COMPILER=Icc
export WM_MPLIB=EASYBUILDMPI
export MPICC=mpicc
export MPICXX=mpiicpc
export CC=icc
export CXX=icpc
export CFLAGS="-no-prec-div -O3 -xHost -fp-model precise -diag-disable 654,1125"
export CXXFLAGS="-O3 -xHost -fp-model precise -diag-disable 654,1125 -DMPICH_SKIP_MPICXX"
export WM_NCOMPPROCS=8 # parallel build
export WM_LABEL_SIZE=32 # 32bit for labels/indices - relevant for METIS (if 64 needed patch METIS first)
# load environment
source OpenFOAM-${VERSION}/etc/bashrc
foam
foamSystemCheck
- build with Intel 2017.1.138: OpenFOAM/config_Intel-2017.1.138
- build with Stage/2016a OpenFOAM/config_2016a
- build with Score-P OpenFOAM/config_ScoreP
- build for Intel/Xeon Phi (Knights Landing) OpenFOAM/config_knl
4. Setup ThirdParty
OpenFOAM has configs which set where it expects to find its "third party" packages.
There are in $FOAM_ETC/config/ as various shell scripts.
The top-level ./Allwmake will build Scotch, but not CGAL, Boost, GMP, and MPFR.
Those packages need to be built separately, and installed into the appropriate directory.
CGAL
Patch $INST_DIR/OpenFOAM-${VERSION}/etc/config/CGAL.sh
### CGAL.sh + boost_version=$EBVERSIONBOOST + cgal_version=$EBVERSIONCGAL + export BOOST_ARCH_PATH=$EBROOTBOOST + export CGAL_ARCH_PATH=$EBROOTCGAL
- Boost -installed as module Boost/1.61.0-Intel-2017.0.098
- GMP - installed as module GMP/6.1.2-Intel-2017.0.098
- MPFR - installed as module MPFR/3.1.5-Intel-2017.0.098
METIS
Patch $INST_DIR/OpenFOAM-${VERSION}/etc/config/metis.sh
# metis.sh + export METIS_ARCH_PATH=$EBROOTMETIS
Patch $WM_THIRD_PARTY_DIR/Allwmake
# $WM_THIRD_PARTY_DIR/Allwmake + if [ -d "$METIS_ARCH_PATH" ] # if [ -d "$METIS_VERSION" ] + then + if [ -f $METIS_ARCH_PATH/include/metis.h \ + -a -r $METIS_ARCH_PATH/lib/libmetis.so ] + then + cp $METIS_ARCH_PATH/lib/libmetis.so $FOAM_EXT_LIBBIN + echo " metis header in $METIS_ARCH_PATH/include" + echo " metis libs in $FOAM_EXT_LIBBIN"
- ATTENTION: metis.h
- METIS library's IDXTYPEWIDTH must fit to $WM_LABEL_SIZE (for METIS/5.1.0-Intel-2017.0.098: IDXTYPEWIDTH=32)
- METIS library's REALTYPEWIDTH must fit to $WM_PRECISION_OPTION (for METIS/5.1.0-Intel-2017.0.098: REALTYPEWIDTH=64)
Scotch
Scotch decomposition which requires no geometric input from the user and attempts to minimise the number of processor boundaries.
Patch $WM_THIRD_PARTY_DIR/etc/wmakeFiles/scotch/Makefile.inc.i686_pc_linux2.shlib-OpenFOAM-64Int32
# scotch/Makefile.inc.i686_pc_linux2.shlib-OpenFOAM-64Int32
+ AR = ${CC}
+ CCS = ${CC}
+ CCP = ${MPICC}
+ CCD = ${MPICC}
+ CFLAGS = -O3 -xHost -fp-model precise -DCOMMON_FILE_COMPRESS_GZ -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict=__restrict
5. Build & Test OpenFOAM
Build OpenFOAM
Prepare for build
# cd $WM_PROJECT_DIR foam # clean if this is a rebuild wcleanAll ; wcleanLnIncludeAll ; wcleanMachine $WM_THIRD_PARTY_DIR/Allclean
Copy third party source to required location.
source $INST_DIR/OpenFOAM-${VERSION}/etc/config/scotch.sh
mkdir -p $SCOTCH_ARCH_PATH
cp -a $WM_THIRD_PARTY_DIR/scotch_6.0.3/* $SCOTCH_ARCH_PATH
Build
# build wmake ./wmake/makeWmake # build OpenFOAM (except Qt & ParaView) ./Allwmake
Test OpenFOAM
Validate the build by running
# cd $WM_PROJECT_DIR foam foamInstallationTest # create the user run directory mkdir -p $FOAM_RUN # test with a simple tutorial: run cp -r $FOAM_TUTORIALS/incompressible/simpleFoam/pitzDaily ./ cd pitzDaily blockMesh simpleFoam # test with a more advanced tutorial # details here: https://cfd.direct/openfoam/user-guide/cavity#x5-40002.1 run cp -r $FOAM_TUTORIALS/incompressible/icoFoam/cavity ./ cd cavity foamRunTutorials #paraFoam
- More detailed tutorials to test can be found here:
6. Build Tools
ParaView
OpenFOAM is shipped with ParaView for post-processing OpenFOAM field results.
This requires a separate compilation which can be omitted if using other post-processing software.
cd $WM_THIRD_PARTY_DIR ./makeParaView
References
- flex 2.6.0
- 2.6.0 causes errors: undefined reference to `yyFlexLexer::yywrap()'
- https://www.cfd-online.com/Forums/openfoam-installation/163986-building-openfoam-3-0-archlinux.html
- -fp-model precise
- Disables optimizations that are not value-safe on floating-point data.
- https://www.nccs.nasa.gov/images/FloatingPoint_consistency.pdf
- -DMPICH_SKIP_MPICXX
- -diag-disable 654
- hide warning #654:
- -diag-disable 1125
- hide warning #1125: function "..." is hidden by "..." -- virtual function override intended?
