Software/OpenFOAM – vis
wiki:Software/OpenFOAM

!!! GENERAL INSTRUCTION - NOT OPTIMIZED FOR PRODUCTION !!!
For production runs please check the installed OpenFOAM on JURECA with

module spider OpenFOAM

OpenFOAM - Build Instructions for JURECA

OpenFOAM is free, open-source software for computational fluid dynamics.

OpenFOAM releases are scheduled every six months in June and December. Releases designated OpenFOAM+ contain client-sponsored developments of which much has been transferred to, but not released in the OpenFOAM Foundation branch.

1. Preliminaries

  • There are, roughly, two components to OpenFOAM:
    • the actual CFD code FOAM -- definitely to be compiled with ICC
    • the GUI analysis tools, based on ParaView (using the Qt GUI toolkit)
  • All OpenFOAM applications do parallel communication through the Pstream library only.
    So using any underlying message passing library (MPI, etc.) is just a matter of replacing Pstream. more...
  • Some of the ThirdParty packages need to be built first. These third party sources are required, but are not bundled. The source tarballs could be expanded in the ThirdParty-${VERSION} directory, but we use installed modules instead.
    • Boost - 1.61.0 works
    • CGAL - 4.8.2 works
      • GMP - 6.1.2 works
      • MPFR - 3.1.5 works
    • METIS - 5.1.0 is expected METIS Official Website
### set env. variables we will use

VERSION=3.0.1

# $INST_DIR must NOT (!!) end with a '/', or build will fail with "touch [...] permission denied"
INST_DIR=<my_openfoam_install_dir> # e.g. ${HOME}/OpenFOAM_install

mkdir -p $INST_DIR

2. Download & Extract

Source packages are available from the download section of the OpenFOAM website:

The source code must be unpacked into the top-level directory of the final installation.
For this HowTo this is $INST_DIR.

### download OpenFOAM
cd $INST_DIR

wget https://downloads.sourceforge.net/project/foam/foam/${VERSION}/OpenFOAM-${VERSION}.tgz
wget https://downloads.sourceforge.net/project/foam/foam/${VERSION}/ThirdParty-${VERSION}.tgz

# extract OpenFOAM
tar -xzf OpenFOAM-${VERSION}.tgz -C $INST_DIR
tar -xzf ThirdParty-${VERSION}.tgz -C $INST_DIR

3. Setup OpenFOAM

Patch

Patch $INST_DIR/OpenFOAM-${VERSION}/etc/bashrc

### bashrc

+ foamInstall=${INST_DIR}
+ : ${WM_COMPILER:=Gcc}; export WM_COMPILER
+ : ${WM_MPLIB:=SYSTEMOPENMPI}; export WM_MPLIB
  • Instead of setting variable in bashrc, they can be set via environment variables before you call bashrc later.
    • export FOAM_INST_DIR=... overwrites foamInstall
    • export WM_COMPILER=..., default is Gcc, should be set to Icc
    • export WM_MPLIB=..., default is SYSTEMOPENMPI, should be set to EASYBUILDMPI

Patch $INST_DIR/OpenFOAM-${VERSION}/etc/config/settings.sh

### settings.sh
Linux)
...
    x86_64)
...
        64)
            WM_ARCH=linux64
            export WM_COMPILER_LIB_ARCH=64
+           export WM_CC=${CC}                      # for use in non-wmake (e.g. ThirdParty) compilations
+           export WM_CXX=${CXX}                    # for use in non-wmake (e.g. ThirdParty) compilations
+           export WM_CFLAGS="${CFLAGS} -fPIC"      # for use in non-wmake (e.g. ThirdParty) compilations
+           export WM_CXXFLAGS="${CXXFLAGS} -fPIC"  # for use in non-wmake (e.g. ThirdParty) compilations
+           export WM_LDFLAGS=                      # for use in non-wmake (e.g. ThirdParty) compilations

### settings.sh

+ #unset MPI_ARCH_PATH MPI_HOME FOAM_MPI_LIBBIN

+ EASYBUILDMPI)
+    export FOAM_MPI=mpi
+    ;;

Patch $INST_DIR/OpenFOAM-${VERSION}/wmake/rules

### linux64Gcc/c, linux64Icc/c
+ cc        = ${MPICC}

### linux64Gcc/c++
+ CC        = ${MPICXX} -std=c++0x 

### linux64Icc/c++
+ CC        = ${MPICXX} -std=c++0x -fp-trap=common -fp-model precise

### linux64Gcc/cOpt, linux64Icc/cOpt
+ cOPT      = ${CFLAGS}
                                 
### linux64Gcc/c++Opt, linux64Icc/c++Opt
+ c++OPT    = ${CXXFLAGS}

Set Environment

The environment must be set up according to where the source code directories are located.

# set environment vars, if not set yet
# $INST_DIR must NOT (!!) end with a '/', or build will fail with "touch [...] permission denied"
#cd <my_openfoam_install_dir>
#export INST_DIR=$(pwd)
#export VERSION=3.0.1
cd $INST_DIR

# load modules
module --force purge
module use /usr/local/software/jureca/OtherStages
module load Stages/Devel
module load Intel/2017.0.098-GCC-5.4.0
module load IntelMPI/2017.0.098
module load flex/2.5.39 # flex 2.6.0 causes errors
module load Boost/1.61.0-Intel-2017.0.098
module load MPFR/3.1.5-Intel-2017.0.098
module load GMP/6.1.2-Intel-2017.0.098
module load CGAL/4.8.2-Intel-2017.0.098
module load METIS/5.1.0-Intel-2017.0.098

# configure environment
export FOAM_INST_DIR=$INST_DIR
export WM_COMPILER=Icc
export WM_MPLIB=EASYBUILDMPI
export MPICC=mpicc
export MPICXX=mpiicpc
export CC=icc
export CXX=icpc
export CFLAGS="-no-prec-div -O3 -xHost -fp-model precise -diag-disable 654,1125"
export CXXFLAGS="-O3 -xHost -fp-model precise -diag-disable 654,1125 -DMPICH_SKIP_MPICXX"

export WM_NCOMPPROCS=8  # parallel build
export WM_LABEL_SIZE=32 # 32bit for labels/indices - relevant for METIS (if 64 needed patch METIS first)

# load environment
source OpenFOAM-${VERSION}/etc/bashrc
foam
foamSystemCheck

4. Setup ThirdParty

OpenFOAM has configs which set where it expects to find its "third party" packages. There are in $FOAM_ETC/config/ as various shell scripts.

The top-level ./Allwmake will build Scotch, but not CGAL, Boost, GMP, and MPFR.
Those packages need to be built separately, and installed into the appropriate directory.

CGAL

Patch $INST_DIR/OpenFOAM-${VERSION}/etc/config/CGAL.sh

### CGAL.sh

+ boost_version=$EBVERSIONBOOST
+ cgal_version=$EBVERSIONCGAL

+ export BOOST_ARCH_PATH=$EBROOTBOOST
+ export CGAL_ARCH_PATH=$EBROOTCGAL
  • Boost -installed as module Boost/1.61.0-Intel-2017.0.098
  • GMP - installed as module GMP/6.1.2-Intel-2017.0.098
  • MPFR - installed as module MPFR/3.1.5-Intel-2017.0.098

METIS

Patch $INST_DIR/OpenFOAM-${VERSION}/etc/config/metis.sh

# metis.sh

+ export METIS_ARCH_PATH=$EBROOTMETIS

Patch $WM_THIRD_PARTY_DIR/Allwmake

# $WM_THIRD_PARTY_DIR/Allwmake

+ if [ -d "$METIS_ARCH_PATH" ] # if [ -d "$METIS_VERSION" ]
+ then
+    if [ -f $METIS_ARCH_PATH/include/metis.h \
+      -a -r $METIS_ARCH_PATH/lib/libmetis.so ]
+    then
+        cp $METIS_ARCH_PATH/lib/libmetis.so $FOAM_EXT_LIBBIN
+        echo "    metis header in $METIS_ARCH_PATH/include"
+        echo "    metis libs   in $FOAM_EXT_LIBBIN"
  • ATTENTION: metis.h
    • METIS library's IDXTYPEWIDTH must fit to $WM_LABEL_SIZE (for METIS/5.1.0-Intel-2017.0.098: IDXTYPEWIDTH=32)
    • METIS library's REALTYPEWIDTH must fit to $WM_PRECISION_OPTION (for METIS/5.1.0-Intel-2017.0.098: REALTYPEWIDTH=64)

Scotch

Scotch decomposition which requires no geometric input from the user and attempts to minimise the number of processor boundaries.

Patch $WM_THIRD_PARTY_DIR/etc/wmakeFiles/scotch/Makefile.inc.i686_pc_linux2.shlib-OpenFOAM-64Int32

# scotch/Makefile.inc.i686_pc_linux2.shlib-OpenFOAM-64Int32
+ AR             = ${CC}
+ CCS            = ${CC}
+ CCP            = ${MPICC}
+ CCD            = ${MPICC}
+ CFLAGS         = -O3 -xHost -fp-model precise -DCOMMON_FILE_COMPRESS_GZ -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict=__restrict

5. Build & Test OpenFOAM

Build OpenFOAM

Prepare for build

# cd $WM_PROJECT_DIR
foam

# clean if this is a rebuild
wcleanAll ; wcleanLnIncludeAll ; wcleanMachine
$WM_THIRD_PARTY_DIR/Allclean

Copy third party source to required location.

source $INST_DIR/OpenFOAM-${VERSION}/etc/config/scotch.sh 

mkdir -p $SCOTCH_ARCH_PATH
cp -a $WM_THIRD_PARTY_DIR/scotch_6.0.3/* $SCOTCH_ARCH_PATH

Build

# build wmake
./wmake/makeWmake

# build OpenFOAM (except Qt & ParaView) 
./Allwmake

Test OpenFOAM

Validate the build by running

# cd $WM_PROJECT_DIR
foam
foamInstallationTest

# create the user run directory
mkdir -p $FOAM_RUN

# test with a simple tutorial:
run
cp -r $FOAM_TUTORIALS/incompressible/simpleFoam/pitzDaily ./
cd pitzDaily
blockMesh
simpleFoam

# test with a more advanced tutorial
# details here: https://cfd.direct/openfoam/user-guide/cavity#x5-40002.1
run
cp -r $FOAM_TUTORIALS/incompressible/icoFoam/cavity ./
cd cavity
foamRunTutorials

#paraFoam 

6. Build Tools

ParaView

OpenFOAM is shipped with ParaView for post-processing OpenFOAM field results.
This requires a separate compilation which can be omitted if using other post-processing software.

cd $WM_THIRD_PARTY_DIR
./makeParaView

References

Last modified 16 months ago Last modified on 08/06/18 19:15:22