View source for Petsc

Petsc is the Portable Extensible Toolkit for Scientific Computation.  It's used for the parallel solution of PDEs and provides parallel linear and nonlinear solvers.  Petsc is written in C.

= Installation =

== Petsc 2.3.3 ==

Petsc 2.3. can be installed using the following configure line.  This will require installation of MPI (I suggest [[LAM MPI]]).

<syntaxhighlight>
/path/to/source/of/petsc-2.3.3/config/configure.py \
   --prefix=/path/to/petsc-2.3.3 \
   --with-matlab=false \
   --with-x=false \
   --with-shared=0 \
   --with-mpi-dir=/path/to/mpi \
   PETSC_DIR=/path/to/source/of/petsc-2.3.3
</syntaxhighlight>

To make this a debug build, add <code>--with-debugging=1</code> to the configure line.

The build process consists of:

# run the configure wrapper 
# turn on lam with lamboot (or whatever MPI you're using)
# make all
# make install
# make test (this script should pass all tests OK)

== Petsc 3.0.0 ==

Petsc 3.0.0 can be installed using the configure line:

<syntaxhighlight>
/path/to/source/of/petsc-3.0.0/config/configure.py \
   --prefix=/path/to/petsc-3.0.0 \
   --with-matlab=false \
   --with-x=false \
   --with-shared=0 \
   --with-mpi-dir=/path/to/mpi \
   --with-clanguage=c++ \
   --with-fc=0 \
   PETSC_DIR=/path/to/source/of/petsc-3.0.0
</syntaxhighlight>

The build process is similar to the above:

# run the configure wrapper 
# turn on lam with lamboot (or whatever MPI you're using)
# make all
# make install
# make test (this script should pass all tests OK)

== Flags explanation ==

The <code>--with-clanguage=c++</code> flag is required if you are building Petsc to be used from a C++ program.  Otherwise, Petsc will try and use a C compiler, which will cause a lot of problems with C++ code.

The <code>--with-fc=0</code> flag turns off Fortran.  I don't use the Fortran interface, and unfortunately Fortran is the cause of many problems on OS X (thank you Apple for your lack of standards).  I leave it out so that I won't have to deal with additional errors related to things I will never use.

= Building with Petsc =

== Source code ==

There are a number of different header files that must be included in any source code that uses Petsc objects or functions.  These must be included by using:

<code>#include "headerfile.h"</code>

Depending on the functionality used, different header files must be used.

{|class="wikitable"
!Header file
!Function

|-
|petsc.h
|generic header file, should be included unless another header file would be more applicable 

|-
|petscvec.h 
|vector structure and functions 

|-
|petscmat.h 
|matrix structure and functions 

|-
|petscksp.h 
|functions solving linear systems using Krylov subspace methods 

|-
|petscpc.h 
|preconditioner structure and functions 

|}

To determine, more specifically, which header file should be included, visit the Petsc documentation page (http://www.mcs.anl.gov/petsc/petsc-as/documentation/index.html), and locate the documentation page for the function or object of interest.  The page will indicate which of the header files listed must be included.

A compiler directive can be used in C or C++ code to check if Petsc is being used:

<syntaxhighlight lang="cpp">
#ifdef HAVE_PETSC 
#include "petsc.h"
#endif
</syntaxhighlight>

This is useful if, for example, you want to build a program that will run with or without Petsc.

== Petsc Makefiles ==

{{Main|Example Petsc Makefile}}

''Note: this section will presume you are building a C++ program.  If you're building a C program, there are many examples provided by Petsc.  These can be found using:''

<syntaxhighlight>
$ cd /path/to/src/of/petsc-3.0.0/src

$ find . -name examples -type d

./contrib/semiLagrange/examples
./dm/adda/examples
./dm/ao/examples
./dm/da/examples
./dm/mesh/examples
./ksp/ksp/examples
./ksp/pc/examples
./ksp/pc/impls/is/feti/examples
./mat/examples
./snes/examples
./sys/draw/examples
./sys/error/examples
./sys/examples
./sys/random/examples
./sys/viewer/examples
./tops/examples
./ts/examples
./vec/is/examples
./vec/pf/examples
./vec/vec/examples

</syntaxhighlight>

''For more on building Makefiles, see [[Make]].

Compilation of a C or C++ program using Petsc is best done with [[Make|Makefiles]], since there is a large amount of information that needs to be fed to the compiler, and Petsc provides that information.

To begin, information contained in variables that make can understand must be included using make's <code>include</code> command:

<syntaxhighlight lang="make">
include ${PETSC_DIR}/conf/base
</syntaxhighlight>

in turn, <code>${PETSC_DIR}/conf/base</code> includes several other files.  The information contained in these files includes linker and compiler commands, <code>-L</code> and <code>-l</code> library flags for compilers, include directories, etc.

A Makefile for a C++ program utilizing Petsc may be constructed using the following structure:

=== Section 1: Variable declaration ===

As shown above, several variables must be declared.  The path to the Petsc directory is useful, so it is put into the <code>$(PETSC_DIR)</code> make macro.  Additionally, other variables are defined by including files provided by Petsc.

<syntaxhighlight lang="make">
PETSC_DIR=/path/to/petsc-3.0.0

include ${PETSC_DIR}/conf/base
</syntaxhighlight>

=== Section 2: Files ===

There are lots of ways to keep your Makefiles short and to the point.  First, a list of files containing source code can be defined:

<syntaxhighlight lang="make">
SRC_FILES = \
    BoundaryConditionFactory.cc \
    BoundaryCondition.cc \
    Field.cc \
    FileIO.cc \
    JacobiSolver.cc \
    Timer.cc \
    TimerFactory.cc \
    GmresSolver.cc \
</syntaxhighlight>

and a make macro can be used to create an analogous list, but replacing all ".cc" suffixes with ".o":

<syntaxhighlight lang="make">
OBJ_FILES = $(SRC_FILES:%.cc=%.o)
</syntaxhighlight>

=== Section 3: Make targets ===

There are a couple of useful make targets one can define.  

The first target should define how to build non-Petsc object files:

<syntaxhighlight lang="make">
%.o: %.cc
    g++ -c -Wall -I. $< -o $@
</syntaxhighlight>

The second target should define how to build Petsc object files.  This example encapsulates all Petsc-related calls in the driver and in a GMRES solver class, so the target is only defined for those file.  If there were more files using Petsc, a list could be constructed, and a make target generated from that rule.

<syntaxhighlight lang="make">
GmresSolver.o: GmresSolver.cc
    ${PETSC_COMPILE} -c -Wall -I. -I$(PETSC_DIR)/inc
</syntaxhighlight>

This rule uses some include directives to point the compiler to the Petsc header files and objects used in the source code file GmresSolver.cc.

Finally, the driver make target, which compiles the driver and links all compiled objects to the driver, requires Petsc libraries to be linked in as well (see the [[Presentations]] page, Scientific Computing Summer Workshop 2, for details).

<syntaxhighlight lang="make">
Laplace: Laplace.cc $(OBJ_FILES)
    -${CLINKER} Laplace.cc -o bin.x $(OBJ_FILES) ${PETSC_LIB} -I$(PETSC_DIR)/include
</syntaxhighlight>

A couple of things to note here.  First, when linking my driver source code, I'm adding <code>${PETSC_LIB}</code> (which adds all of the library links that I will need) because I've got an <code>#include "petsc.h"</code> and <code>#include "petscksp.h"</code> in my driver source code (the <code>${PETSC_LIB}</code> variable includes all Petsc libraries during linking, so it's good to use it instead of, say, <code>${PETSC_KSP_LIB}</code>, just to be safe).  Second, I have to link everything using <code>-${CLINKER}</code> (minus sign and all).  If I were to compile Petsc to use C, then <code>$CLINKER</code> would point to <code>mpicc</code>, or whatever MPI C-compiler my MPI distribution happened to provide.  However, using the configure flag <code>--with-clanguage=c++</code> makes the variable <code>${CLINKER}</code> point to <code>mpic++</code>, or whatever C++-compiler my MPI distribution happens to provide.

= Using Petsc in C++ =




= See Also =

* [[C++]]
* [[Example Petsc Makefile]]
* [[Hypre]]
* [[Make]]



* [[CFD Lectures|Computational fluid dynamics]] (lecture notes)
* [[PDE Lectures|Partial differential equations]] (lecture notes)




[[Category:Programs]]
[[Category:Computers]]