Parallel FTCS algorithm
Strategy
We split the domain into vertical strips. Each strip has a local x-index running from i=0 to i=ni+1 with i=0, i=ni+1 corresponding to regions outside of the strip. The values for these must be obtained from the adjoining strips or the boundary conditions.
We accomplish this by sending messages between the processes with the values in these ghost cells. The communication is structured in the following way:
All strips except the last send their i=ni gridline values. These correspond to the i=0 ghost cell values for the right-adjoining strip.
All strips except the first wait to receive their i=0 ghost cell values.
All strips except the first send their i=1 gridline values. These correspond to the i=ni+1 ghost cell values for the left-adjoining strip.
All strips except the last wait to receive their i=ni+1 ghost cell values
You have to be careful about how full the last grid strip is but other than that the parallel implementation is pretty straightforward.
CALL MPI_COMM_SIZE(MPI_COMM_WORLD,np,ierr)
CALL
MPI_COMM_RANK(MPI_COMM_WORLD,idproc,ierr)
DO WHILE (t<tfinal)
DO j=1,ny
DO i=1,MIN(ni,nx+1-idproc*ni)
qnew(i,j)=qold(i,j)+sigmax*(qold(i+1,j)-2*qold(i,j)+qold(i-1,j)) &
+sigmay*(qold(i,j+1)-2*qold(i,j)+qold(i,j-1))
END DO
END DO
IF (idproc < np-1) THEN
qright(:)=qnew(ni,:)
CALL MPI_SEND(qright(0), ny+1,
MPI_REAL, idproc+1, TAG_DATASYNCH, MPI_COMM_WORLD, ierr)
END IF
IF (idproc > 0) THEN
CALL MPI_RECV(qleft(0), ny+1,
MPI_REAL, idproc-1, TAG_DATASYNCH, MPI_COMM_WORLD, ierr)
qnew(0,:)=qleft(:)
END IF
IF (idproc > 0) THEN
qleft(:)=qnew(1,:)
CALL MPI_SEND(qleft(0), ny+1,
MPI_REAL, idproc-1, TAG_DATASYNCH, MPI_COMM_WORLD, ierr)
END IF
IF (idproc < np-1) THEN
CALL MPI_RECV(qright(0), ny+1,
MPI_REAL, idproc+1, TAG_DATASYNCH, MPI_COMM_WORLD, ierr)
qnew(ni+1,:)=qright(:)
END IF
qold=qnew
t=t+delt
Basic MPI Programming
Making MPI defintions accessible to your program
Making the MPI definitions accessible to your program
MPI defines a large number of system wide data structures and constants. To make your program aware of these always begin your code with:
INCLUDE 'mpif.h'
MPI initialization and shutdown
Before any call to the MPI library may be made the system must be initialized by:
CALL MPI_INIT(ierr)
ierr - INTEGER error code with the result of the initialization
After completing the computation a clean shutdown of the MPI system should be achieved by calling:
CALL MPI_FINALIZE(ierr)
Finding information about the environment
In a simple MPI program a single code is run on multiple processes. Typically the code uses a 'master' process that provides overall control and 'drone' or 'slave' processes that carry out computational tasks. All MPI processes have an integer identification number (idproc) running from 0 to np-1, where np is the number of processes running. By convention idproc=0 is the master process.
To find out how many processes are active:
CALL MPI_COMM_SIZE(MPI_COMM_WORLD,np,ierr)
To find out the ID of the particular process running the code at present:
CALL MPI_COMM_RANK(MPI_COMM_WORLD,idproc,ierr)
MPI_COMM_WORLD is defined in mpif.h.
Basic Communication
Processes communicate information by sending and receiving messages. The MPI routine that sends a message is invoked by
CALL MPI_SEND(buf, count, type, iddest, itag, MPI_COMM_WORLD, ierr)
where:
buf - a buffer holding the data to be sent
count - the number of items in the buffer
type - the type of the items in the buffer. Common predefined types include MPI_INTEGER, MPI_REAL, MPI_DOUBLE_PRECISION, MPI_COMPLEX, MPI_LOGICAL with obvious Fortran equivalents.
iddest - the ID of the destination process.
itag - an integer tag attached to the message for identification purposes
The MPI routine that receives a message is invoked by
CALL MPI_RECV(buf, count, type, idsrc, itag, MPI_COMM_WORLD, ierr)
where:
buf - a buffer to hold the received data
count - the number of items in the buffer
type - the type of the items in the buffer.
idsrc - the ID of the process from which to receive the message. A special value MPI_ANY_SOURCE allows reception from any process.
itag - an integer tag attached to the message for identification purposes. A special value MPI_ANY_TAG allows reception of a message irrespective of its tag.
Timing
Of course we'd like to know how parallel techniques are cutting down the wall clock time. Use the function
twall=MPI_WTIME()
to find the current time in seconds (twall should be DOUBLE PRECISION). You'll probably want only the master process to keep track of elapsed time.
Compiling and running an MPI Program
These instructions are specific to LAM-MPI.
Starting/Shutting down the MPI system
Before running a parallel application the MPI system must be initialized by:
lamboot -v <bhost.def>
The optional <bhost.def> argument specifies the boot schema. If it is not specified the MPI system first looks for a file bhost.def in the current directory, then in the MPI home directory. One can use only a part of a cluster by creating a new bhost.def file and then specifying the bhost.def file on the lamboot command line.
After completing your MPI session type:
wipe -v
to shut down the MPI system.
Compiling a parallel application
To compile a Fortran application using MPI type:
hf77 -o <yourapp> <yoursrc.f90> -lmpi
for Fortran 90/95 code or
hf77 -o <yourapp> <yoursrc.f> -lmpi
for Fortran 77 code.
Running a parallel application
To run a parallel application using the LAM MPI implementation type:
mpirun -np <n> -v <yourapp>
Parameters:
- n is the number of processes used in the run. Processes get automatically mapped to processors by the operating system.
- v is a switch telling the MPI system to print useful information about the steps it is taking