Networks of Workstations: (Distributed Memory)

Networks of
Workstations
(distributed memory)
Parallel and Distributed Computing
Arfan Shahzad
{ arfanskp@gmail.com }
Course Outlines
Workstation
• Workstation, a high-performance computer system than mainstream personal

computers, that is basically designed for a single user and has advanced graphics
capabilities, large storage capacity, and a powerful central processing unit.
• A workstation is more capable than a personal computer (PC) but is less

advanced than a server (which can manage a large network of peripheral PCs or
workstations and handle immense data-processing and reporting tasks).
Workstation cont…
• Workstations were the first segment of the computer market to
present advanced accessories and collaboration tools.
• The increasing capabilities of mainstream PCs in the late 1990s have
blurred the lines between PCs and technical/scientific workstations.

Network of Workstations (NOW)
• High-speed networks and rapidly improving microprocessor
performance make networks of workstations an increasingly
appealing vehicle for parallel computing.
• By relying solely on commodity hardware and software, networks of
workstations offer parallel processing at a relatively low cost.

Network of Workstations (NOW) cont…
commodity hardware and software
• Commodity hardware (unlike purpose-built hardware), sometimes
known as off-the-shelf hardware, is a computer device or IT
component that is relatively inexpensive, widely available and
basically interchangeable with other hardware of its type.

• A network-of-workstations multiprocessor may be realized as a
processor bank, a number of processors dedicated for the purpose of
providing computing cycles.
• Alternatively, it may consist of a dynamically varying set of machines
on which idle cycles are used to perform long-running computations.

• In the latter case, the (hardware) cost is essentially zero, since many
organizations already have extensive workstation networks in place.
• In terms of performance, improvements in processor speed and

network bandwidth and latency allow networked workstations to
deliver performance approaching or exceeding supercomputer
performance for an increasing class of applications.
Distributed Memory
• Distributed memory refers to a multiprocessor computer system in
which each processor has its own private memory.
• Computational tasks can only operate on local data, and if remote
data are required, the computational task must communicate with
one or more remote processors.

Distributed Memory
Distributed Memory
• In a distributed memory system there is typically a processor, a
memory, and some form of interconnection that allows programs on
each processor to interact with each other.

Distributed Memory (programming issues)
• The key issue in programming distributed memory systems is how to
distribute the data over the memories.
• Depending on the problem solved, the data can be distributed statically, or

it can be moved through the nodes.
• Data can be moved on demand, or data can be pushed to the new nodes
in advance.
• Data can be kept statically in nodes if most computations happen
locally, and only changes on edges have to be reported to other
nodes.
• An example of this is simulation where data is modeled using a grid

(non-interactive workloads), and each node simulates a small part of
the larger grid.
• On every iteration, nodes inform all neighboring nodes of the new
edge data.
Shared vs. Distributed Memory cont…
• There are two kinds of multiple-processor systems exist:
• Multi-Processors (Shared/ Distributed)
• Multi-Computers (Distributed)
• In a multi-processor two or more CPUs share a common main

memory.
• Any process on any processor, can read or write any word in shared
memory, simply by moving data to or from the desired location.
• In a multi-computer, in contrast, each CPU has its own private

memory. Nothing is shared.
• To make an agriculture
analogy, a multiprocessor is a
system with a herd of sheep
(processes) eating from a
single feeding through
(shared memory).
• A multicomputer, on the other hand, is a design in which each sheep

has its own private feeding through (distributed memory).

Networks of Workstations: (Distributed Memory)

Uploaded by

Copyright:

Available Formats

Networks of Workstations: (Distributed Memory)

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Networks of Workstations: (Distributed Memory)

Uploaded by

Copyright:

Available Formats

Networks of

• Workstation, a high-performance computer system than mainstream personal

• A workstation is more capable than a personal computer (PC) but is less

• Workstations were the first segment of the computer market to

present advanced accessories and collaboration tools.

• The increasing capabilities of mainstream PCs in the late 1990s have

blurred the lines between PCs and technical/scientific workstations.

• High-speed networks and rapidly improving microprocessor

performance make networks of workstations an increasingly

appealing vehicle for parallel computing.

• By relying solely on commodity hardware and software, networks of

workstations offer parallel processing at a relatively low cost.

known as off-the-shelf hardware, is a computer device or IT

component that is relatively inexpensive, widely available and

basically interchangeable with other hardware of its type.

• A network-of-workstations multiprocessor may be realized as a

processor bank, a number of processors dedicated for the purpose of

providing computing cycles.

• Alternatively, it may consist of a dynamically varying set of machines

on which idle cycles are used to perform long-running computations.

• In terms of performance, improvements in processor speed and

which each processor has its own private memory.

• Computational tasks can only operate on local data, and if remote

data are required, the computational task must communicate with

one or more remote processors.

memory, and some form of interconnection that allows programs on

each processor to interact with each other.

• Depending on the problem solved, the data can be distributed statically, or

• An example of this is simulation where data is modeled using a grid

• There are two kinds of multiple-processor systems exist:

• Multi-Processors (Shared/ Distributed)

• In a multi-processor two or more CPUs share a common main

• In a multi-computer, in contrast, each CPU has its own private

• A multicomputer, on the other hand, is a design in which each sheep

You might also like

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.