Scribd
Upload
5 views33 pages

MPI - Python Lab

The document provides an overview of MPI (Message Passing Interface) and its implementation in Python using the mpi4py module, covering concepts such as multithreading, multiprocessing, and high-performance computing. It details the installation process, key functions for point-to-point and collective communication, and examples of various operations including broadcasting, scattering, gathering, and reduction. Additionally, it discusses the SPMD model and strategies to avoid deadlocks in communication.

Uploaded by

CS INSIDE FI
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
5 views33 pages

MPI - Python Lab

The document provides an overview of MPI (Message Passing Interface) and its implementation in Python using the mpi4py module, covering concepts such as multithreading, multiprocessing, and high-performance computing. It details the installation process, key functions for point-to-point and collective communication, and examples of various operations including broadcasting, scattering, gathering, and reduction. Additionally, it discusses the SPMD model and strategies to avoid deadlocks in communication.

Uploaded by

CS INSIDE FI
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 33

MPI - Python

Basic definitions
What is Multithreading?
• A single core(processor) in your computer can mimic multi(N)processor, This will allow you to run
multiple(N) programs concurrently(not exactly!).
• Multiple processor can share same memory space. So that a single variable can be updated or used by
multiple processor at the same time.
What is Multiprocessing?
• A single core(processor) is always single. So, for running program concurrently(exactly), you need to
use multiple physical cores(processors)
• Multiple processor doesn't share memory. So, we need to make them talk among themselves.
High Performance Computing(HPC)
• Uses both of this concept efficiently
Topics covered
• Using the mpi4py Python module
• Implementing point-to-point communication
• Avoiding deadlock problems
• Collective communication using a broadcast
• Collective communication using the scatter function
• Collective communication using the gather function
• Collective communication using Alltoall
• The reduction operation
• Optimizing communication
Installation Procedure
• The installation procedure of mpi4py on a Windows machine is as
follows:
• C:>pip install mpi4py
• Anaconda users must type the following:
• C:>conda install mpi4py
• This implies that the notation used to run the mpi4py examples
is as follows:
• C:>mpiexec -n x python mpi4py_script_name.py
• The mpiexec command is the typical way to start parallel jobs: x
is the total number of processes to use, while
mpi4py_script_name.py is the name of the script to be
executed.
Strengths of MPI
• Standardization: It is supported by all High-Performance
Computing (HPC) platforms.
• Portability: The changes applied to the source code are
minimal, which is useful if you decide to use the application
on a different platform that also supports the same standard.
• Performance: Manufacturers can create implementations
optimized for a specific type of hardware and get better
performance.
• Functionality: Over 440 routines are defined in MPI-3, but
many parallel programs can be written using fewer than
even 10 routines.
MPI basic functions
• Import the mpi4py library:
from mpi4py import MPI
• If we have a number (p of processes) that runs a program, then the
processes will have a rank that goes from 0 to p-1. In particular, in
order to assess the rank of each process, we must use the
COMM_WORLD MPI function in particular. This function is called a
communicator, as it defines its own set of all processes that can
communicate together:
comm = MPI.COMM_WORLD
• Finally, the following Get_rank() function returns rank of the
process calling it:
rank = comm.Get_rank()
SPMD model
• MPI belongs to the Single Program Multiple Data (SPMD) programming technique.

• SPMD is a programming technique in which all processes execute the same program, each

on different data. The distinction in executions between different processes occurs by


differentiating the flow of the program, based on the local rank of the process.

• SPMD is a programming technique in which a single program is executed by several

processes at the same time, but each process can operate on different data.

• At the same time, the processes can execute both the same instruction and different

instructions.
Modules of MPI
• Point-to-point communication
• Collective communication
• Topologies
Point-to-Point Communication
The mpi4py Python module enables point-to-point communication via two functions:

• Comm.Send(data, process_destination): This function sends data to the


destination process identified by its rank in the communicator group.

• Comm.Recv(process_source): This function receives data from the sourcing


process, which is also identified by its rank in the communicator group.

The Comm parameter, which is short for communicator, defines the group of
processes that

• may communicate through message passing using comm = MPI.COMM_WORLD


Point-to-Point Communication
Point-to-Point Communication
• 3 arguments for communication
(source=0, dest=1, tag=1001)
source = From where you are communicating?
dest = To whom you are communicating?
tag = Are you communicating to me or to someone
else?
• Point to Point types
• Blocking
• Non-Blocking
Example 1
Example 2
Example 3
from mpi4py import MPI if rank==1:

comm=MPI.COMM_WORLD destination_process = 8
data= "hello"
rank = comm.Get_rank()
print("my rank is : " , rank) comm.send(data,dest=destination_process)
if rank==0: print ("sending data %s :" %data + "to
data= 10000000 process %d"
%destination_process)
destination_process = 4 if rank==4:
data=comm.recv(source=0)
comm.send(data,dest=destination_proc
print ("data received is = %s" %data)
ess)
if rank==8:
print ("sending data %s " %data
+ "to data1=comm.recv(source=1)
print ("data1 received is = %s"
process %d" %data1)
%destination_process)
mpiexec -n 9 python
pointToPointCommunication.py
Example 3 - Output
This is the output that you'll get after you run
the script:
my rank is : 7
my rank is : 5
my rank is : 2
my rank is : 6
my rank is : 3
my rank is : 1
sending data hello :to process 8
my rank is : 0
sending data 10000000 to process 4
my rank is : 4
data received is = 10000000
my rank is : 8
data1 received is = hello
Example 4
Blocking Functions
The comm.send() and comm.recv() functions are blocking functions,
which means that they block the caller until the buffered data involved
can be used safely.

• Buffered mode: The flow control returns to the program as soon as


the data to be sent has been copied to a buffer. This does not mean
that the message is sent or received.

• Synchronous mode: The function only gets terminated when the


corresponding receive function begins receiving the message
Avoid Deadlocks in point-point
comm
from mpi4py import MPI
comm=MPI.COMM_WORLD
rank = comm.rank
print("my rank is : " , rank)
if rank==0:
data_send= "a"
destination_process = 1
source_process = 1
data_received=comm.recv(source=source_process)
comm.send(data_send,dest=destination_process)
print ("sending data %s " %data_send + "to process %d" %destination_process)
print ("data received is = %s" %data_received)
Avoid Deadlocks in point-point
comm
if rank==1:
data_send= "b"
destination_process = 0
source_process = 0
comm.send(data_send,dest=destination_process)
data_received=comm.recv(source=source_process)
print ("sending data %s :" %data_send + "to process %d“
%destination_process)
print ("data received is = %s" %data_received)
send_recv() – to avoid deadlocks
if rank==1:
data_send= "a"
destination_process = 5
source_process = 5
data_received=comm.sendrecv(data_send,dest=destination_process,
source =source_process)
if rank==5:
data_send= "b“
destination_process = 1
source_process = 1
data_received=comm.sendrecv(data_send,dest=destination_process,
source=source_process)
Collective communication
routines
The most commonly collective operations are:
• Barrier synchronization across the group's
processes
• Communication functions:
• Broadcasting data from one process to all
process in the group
• Gathering data from all process to one process
• Scattering data from one process to all process
• Reduction operation
Broadcast Example
Broadcast Example
from mpi4py import MPI
comm = MPI.COMM_WORLD
rank = comm.Get_rank()
if rank == 0:
variable_to_share = 100
else:
variable_to_share = None
variable_to_share = comm.bcast(variable_to_share, root=0)
print("process = %d" %rank + " variable shared = %d " %variable_to_share)
Scatter Example
Scatter Example
from mpi4py import MPI
comm = MPI.COMM_WORLD
rank = comm.Get_rank()
if rank == 0:
array_to_share = [1, 2, 3, 4 ,5 ,6 ,7, 8 ,9 ,10]
else:
array_to_share = None
recvbuf = comm.scatter(array_to_share, root=0)
print("process = %d" %rank + " recvbuf = %d " % recvbuf)
Scatter Example
• The mpi4py library provides two other functions that are used to
scatter data:
• comm.scatter(sendbuf, recvbuf, root=0): This sends data from
one process to all other processes in a communicator.
• comm.scatterv(sendbuf, recvbuf, root=0): This scatters data
from one process to all other processes in a group that provides
different amount of data and displacements at the sending side.
• The sendbuf and recvbuf arguments must be given in terms of a list
(as in, the point-topoint function comm.send):
• buf = [data, data_size, data_type]
• Here, data must be a buffer-like object of the size data_size and of
the type data_type.
Gather Example
Gather Example
from mpi4py import MPI
comm = MPI.COMM_WORLD
size = comm.Get_size()
rank = comm.Get_rank()
data = (rank+1)**2
data = comm.gather(data, root=0)
if rank == 0:
print ("rank = %s " %rank + "...receiving data to other process")
for i in range(1,size):
data[i] = (i+1)**2
value = data[i]
print(" process %s receiving %s from process %s" %(rank , value , i))
Reduce Example
• comm.Reduce(sendbuf, recvbuf, rank_of_root_process, op = type_of_
reduction_operation)
• Some of the reduction operations defined by MPI are:
• MPI.MAX: This returns the maximum element
• MPI.MIN: This returns the minimum element
• MPI.SUM: This sums up the elements
• MPI.PROD: This multiplies all elements
• MPI.LAND: This performs a logical operation and across the elements
• MPI.MAXLOC: This returns the maximum value and the rank of the
process that owns it
• MPI.MINLOC: This returns the minimum value and the rank of the
process that owns it
Reduce Example
import numpy
import numpy as np
from mpi4py import MPI
comm = MPI.COMM_WORLD
size = comm.size
rank = comm.rank
array_size = 3
recvdata = numpy.zeros(array_size,dtype=numpy.int)
senddata = (rank+1)*numpy.arange(array_size,dtype=numpy.int)
print(" process %s sending %s " %(rank , senddata))
comm.Reduce(senddata,recvdata,root=0,op=MPI.SUM)
print ('on task',rank,'after Reduce: data = ',recvdata)
Reduce Example
All to All Example
All to All Example
from mpi4py import MPI
import numpy
comm = MPI.COMM_WORLD
size = comm.Get_size()
rank = comm.Get_rank()
a_size = 1
senddata = (rank+1)*numpy.arange(size,dtype=int)
recvdata = numpy.empty(size*a_size,dtype=int)
comm.Alltoall(senddata,recvdata)
print(" process %s sending %s receiving %s"%(rank , senddata , recvdata))

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy