Petsc

Petsc is the Portable Extensible Toolkit for Scientific Computation. It's used for the parallel solution of PDEs and provides parallel linear and nonlinear solvers. Petsc is written in C.

Installation

Petsc 2.3.3

Petsc 2.3. can be installed using the following configure line. This will require installation of MPI (I suggest LAM MPI).

/path/to/source/of/petsc-2.3.3/config/configure.py \
   --prefix=/path/to/petsc-2.3.3 \
   --with-matlab=false \
   --with-x=false \
   --with-shared=0 \
   --with-mpi-dir=/path/to/mpi \
   PETSC_DIR=/path/to/source/of/petsc-2.3.3

To make this a debug build, add --with-debugging=1 to the configure line.

The build process consists of:

1. run the configure wrapper
2. turn on lam with lamboot (or whatever MPI you're using)
3. make all
4. make install
5. make test (this script should pass all tests OK)

Petsc 3.0.0

Petsc 3.0.0 can be installed using the configure line:

/path/to/source/of/petsc-3.0.0/config/configure.py \
   --prefix=/path/to/petsc-3.0.0 \
   --with-matlab=false \
   --with-x=false \
   --with-shared=0 \
   --with-mpi-dir=/path/to/mpi \
   --with-clanguage=c++ \
   --with-fc=0 \
   PETSC_DIR=/path/to/source/of/petsc-3.0.0

The build process is similar to the above:

1. run the configure wrapper
2. turn on lam with lamboot (or whatever MPI you're using)
3. make all
4. make install
5. make test (this script should pass all tests OK)

Flags explanation

The --with-clanguage=c++ flag is required if you are building Petsc to be used from a C++ program. Otherwise, Petsc will try and use a C compiler, which will cause a lot of problems with C++ code.

The --with-fc=0 flag turns off Fortran. I don't use the Fortran interface, and unfortunately Fortran is the cause of many problems on OS X (thank you Apple for your lack of standards). I leave it out so that I won't have to deal with additional errors related to things I will never use.

Building with Petsc

Source code

There are a number of different header files that must be included in any source code that uses Petsc objects or functions. These must be included by using:

#include "headerfile.h"

Depending on the functionality used, different header files must be used.

Header file	Function
petsc.h	generic header file, should be included unless another header file would be more applicable
petscvec.h	vector structure and functions
petscmat.h	matrix structure and functions
petscksp.h	functions solving linear systems using Krylov subspace methods
petscpc.h	preconditioner structure and functions

To determine, more specifically, which header file should be included, visit the Petsc documentation page (http://www.mcs.anl.gov/petsc/petsc-as/documentation/index.html), and locate the documentation page for the function or object of interest. The page will indicate which of the header files listed must be included.

A compiler directive can be used to check if Petsc is being used:

#ifdef HAVE_PETSC 
#include "petsc.h"
#endif

Petsc Makefiles

PETSC_DIR=/Users/charles/pkg/petsc-3.0.0
include ${PETSC_DIR}/conf/base

INCLUDES = -I. -I$(PETSC_DIR) -I$(PETSC_DIR)/bmake-darwin9.5.0-c-opt -I$(PETSC_DIR)/include

SRC_FILES = \
    BoundaryConditionFactory.cc \
    BoundaryCondition.cc \
    Field.cc \
    FileIO.cc \
    JacobiSolver.cc \
    Timer.cc \
    TimerFactory.cc \
    GmresSolver.cc \

OBJ_FILES = $(SRC_FILES:%.cc=%.o)

Finally, the make targets must be defined. The

%.o: %.cc
    g++ -c -Wall -I. $< -o $@

GmresSolver.o: GmresSolver.cc
    g++ -c -Wall -I. -I$(PETSC_DIR) -I$(PETSC_DIR)/bmake-darwin9.5.0-c-opt -I$(PETSC_DIR)/include $< -o $@

#GmresSolver.o: GmresSolver.cc
#   g++ -c -Wall ${INCLUDES} $< -o $@

objects: $(OBJ_FILES)

Laplace: Laplace.cc $(OBJ_FILES)
    -${CLINKER} Laplace.cc -o bin.x $(OBJ_FILES) ${PETSC_KSP_LIB} -I$(PETSC_DIR)/include

this: $(OBJ_FILES)
    -@true

The most important target to notice is the driver target:

Laplace: Laplace.cc $(OBJ_FILES)
    -${CLINKER} Laplace.cc -o bin.x $(OBJ_FILES) ${PETSC_KSP_LIB} -I$(PETSC_DIR)/include

A couple of things to note here. First, when linking my driver source code, I'm adding ${PETSC_KSP_LIB} (which adds all of the library links that I will need) because I've got an #include "petscksp.h" in my driver source code. Second, I have to link everything using -${CLINKER} (minus sign and all). If I were to compile Petsc to use C, then $CLINKER would point to mpicc, or whatever MPI C-compiler my MPI distribution happened to provide. However, using the configure flag --with-clanguage=c++ makes the variable ${CLINKER} point to mpic++, or whatever C++-compiler my MPI distribution happens to provide.