Distributed System Models

Most concepts are

drawn from Chapter 2
Pearson Education
Some ideas from
Chapter 1

Introduction

Real world systems should be designed to

function correctly in ALL circumstances/scenarios.
Distributed system models helps in

..classifying and understanding different implementations

..identifying their weaknesses and their strengths
..crafting new systems outs of pre-validated building blocks

We will study distributed system models from

different perspectives

Structure, organization, and placement of components

Interactions
Fundamental properties of systems

Characterization

The structure and the organization of systems and

the relationship among their components should
be designed with the following goals in mind:

To cover the widest possible range of circumstances.

To face the possible difficulties and threats.
To meet the current and possibly the future demands.

Architectural models provide both:

a pragmatic starting point

a conceptual view

In terms of implementation models and

basic blocks
In terms of logical view of the system,
interaction flow, and components

to address these challenges.

Characterization: Challenges (Difficulties and

Threats)

Widely varying models of use

Wide range of system environments

Heterogeneous hardware, operating systems, network, and

performance.

Internal problems

High variation of workload, partial disconnection of components,

or poor connection.

Non synchronized clocks, conflicting updates, various hardware

and software failures.

External threats

Attacks on data integrity, secrecy, and denial of service.

Characterization: Dealing with Challenges

Observations

Widely varying models of use

Wide range of system environments

A flexible and modular structure allows for implementing different

solutions for different hardware, OS, and networks.

Internal problems

The structure and the organization of systems allow for distribution of

workloads, redundant services, and high availability.

The relationship between components and the patterns of interaction

can resolve concurrency issues, while structure and organization of
component can support failover mechanisms.

External threats

Security has to be built into the infrastructure and it is fundamental for

shaping the relationship between components.

Architectural model

An Architectural model of a distributed system

is concerned with the placement of its parts
and relationship between them. Examples:

Client-Server (CS) and peer process models.

CS can be modified by:

The partitioning of data/replication at cooperative

servers
The caching of data by proxy servers or clients
The use of mobile code and mobile agents
The requirements to add or remove mobile devices.

Fundamental Models

Fundamental Models are concerned with a formal

description of the properties that are common in all
of the architectural models
Models addressing time synchronization, message
delays, failures, security issues are addressed as:

Interaction Model deals with performance and the

difficulty of setting of time limits in a distributed system.
Failure Model specification of the faults that can be
exhibited by processes
Secure Model discusses possible threats to processes
and communication channels.

Physical Models

Explicit in the 5th Edition of CDK

book

Physical Models

A representation of the underlying h/w elements

of a DS that abstracts away specific details of the
computer/networking technologies.
Baseline physical model
3 Generations of DS:

Early distributed systems [late 70-80s]: LAN-based

Internet-scale distributed systems [early 90-2005]:
Clusters, grids, P2P, Clouds
Contemporary distributed systems: dynamic nodes
like mobile-based services (nodes are very dynamic
not static like other models).

Architectural Models

Software Layers
System Architectures
Interfaces and Objects
Design Requirements

Architectural Models Intro [1]

The architecture of a system is its structure in terms

of separately specified components.

Its goal is to meet present and likely future demands.

Major concerns are making the system reliable,
manageable, adaptable, and cost-effective.

Architectural Model:

Simplifies and abstracts the functions of individual

components
The placement of the components across a network of
computers define patterns for the distribution of data and
workloads
The interrelationship between the components ie.,
functional roles and the patterns of communication
between them.

Architectural Models Intro [2]

Architectural Model - simplifies and abstracts

the functions of individual components:

An initial simplification is achieved by classifying

processes as:

Server processes
Client processes
Peer processes

client
client

server
server

Cooperate and communicate in a symmetric manner to

perform a task.
peer
peer
peer
peer

Software Architecture and Layers

The term software architecture referred:

Originally to the structure of software as layers or modules in a single computer.

More recently in terms of services offered and requested between processes in the
same or different computers.

Breaking up the complexity of systems by designing them through layers and

services

Layer: a group of related functional components

Service: functionality provided to the next layer.
Layer N

Layer 2
(services offered to above layer)
Layer 1

Software and hardware service layers in

distributed systems

Applications, services

Middlew are

Operating system
Platform
Computer and netw ork hardw are

Platform

The lowest hardware and software layers are often

referred to as a platform for distributed systems and
applications.
These low-level layers provide services to the layers
above them, which are implemented independently
in each computer.
Major Examples

Intel x86/Windows
Intel x86/Linux
Intel x86/Solaris
SPARC/SunOS
PowerPC/MacOS

Middleware

A layer of software whose purpose is to mask heterogeneity present

in distributed systems and to provide a convenient programming
model to application developers.
Major Examples:

Sun RPC (Remote Procedure Calls)

OMG CORBA (Common Object Request Broker Architecture)
Microsoft D-COM (Distributed Components Object Model)
Sun Java RMI
Modern Middleware:

Manjrasoft Aneka for Cloud computing

IBM WebSphere
Microsoft .NET
Sun J2EE
Google AppEngine

System Architecture

The most evident aspect of DS design is the

division of responsibilities between system
components (applications, servers, and other
processes) and the placement of the
components on computers in the network.
It has major implication for:

Performance, reliability, and security of the

resulting system.

Client Server Basic Model:

Clients invoke individual servers

Client

invocation
result

Client

invocation

Server

result

Key:
Process:

Server

Computer:

Client processes interact with individual server processes in a separate computer

in order to access data or resource. The server in turn may use services of other
servers.
Example:

A Web Server is often a client of file server.

Browser search engine -> crawlers other web servers.

Client-Server Architecture Types

Two-tier model (classic)

client
client

server
server

Three-tier (when the server, becomes a client)

client
client

server
server

Server/client
Server/client

server
server

Multi-tier (cascade model)

client
client

Server/client
Server/client

Server/client
Server/client

server
server

Clients and Servers

General interaction between a client and a server.

A service provided by multiple servers

Service

Server
Client

Server

Client
Server

Services may be implemented as several server processes in separate host computers.

Example: Cluster based Web servers and apps such as Google, parallel databases Oracle

Proxy servers (replication transparency) and

caches: Web proxy server
Web
server

Client
Proxy
server

Client

Web
server

A cache is a store of recently used data.

Peer Processes: A distributed application

based on peer processes
Peer 2
Peer 1

Application

Application
Peer 3

Sharable
objects

Application

Peer 4
Application
Peers 5 .... N

All of the processes play similar roles, interacting cooperatively as peers to

perform distributed activities or computations without distinction between clients
and servers. E.g., music sharing systems Gnutella, Napster, Kaza, etc.
Distributed white board users on several computers to view and interactively
modify a picture between them.

P2P with a Centralized Index Server

(e.g. Napster Architecture)

peer
peer
peer
peer

peer
peer
peer
peer

peer
peer
peer
peer
peer
peer

Variants of Client Sever Model: Mobile code

and Web applets
a) client request results in the dow nloading of applet code

Client
Applet code

Web
server

b) client interacts w ith the applet

Client

Applet

Web
server

Applets downloaded to clients give good interactive response

Mobile codes such as Applets are potential security threat, so the
browser gives applets limited access to local resources (e.g. NO
access to local/user file system).

Variants of Client Sever Model: Mobile Agents

A running program (code and data) that travels from one

computer to another in a network carrying out an autonomous
task, usually on behalf of some other process
advantages: flexibility, savings in communications cost
virtual markets, software maintain on the computers within an
organisation.
Potential security threat to the resources in computers they visit.
The environment receiving agent should decide which of the local
resource to allow. (e.g., crawlers and web servers).
Agents themselves can be vulnerable they may not be able to
complete task if they are refused access.

Thin clients and compute servers

Compute server

Network computer or PC
Thin
Client

network

Application
Process

Network computer: download OS and applications from the

network and run on a desktop (solve up-gradation problem) at
runtime.
Thin clients: Windows-based UI on the user machine and
application execution on a remote computer. E.g, X-11 system.

Mobile devices and spontaneous networking

[3rd Generation Distributed System]

The world is increasingly populated by small and portable

computing devices.
W-LAN needs to handle constantly changing heterogeneous,
roaming devices
Need to provide discovery services: (1) registration service to
enable servers to publish their services and (2) lookup service to
allow clients to discover services that meet their requirements.

Interfaces and Objects

The use of CS has impact on the software

architecture followed:

Basic CS model, responsibility is statically allocated.

Distribution of responsibilities
Synchronization mechanisms between client and server
Admissible types of requests/responses
File server is responsible for file, not for web pages.

Peer process model, responsibility is dynamically

allocated:

In fully decentralized music file sharing system, search

process may be delegated to different peers at runtime.

Design Requirements/Challenges of Distributed

Systems

Performance Issues
Responsiveness

Support interactive clients

Use caching and replication

Throughput
Load balancing and timeliness

Quality of Service:
Reliability
Security
Adaptive performance.
Dependability issues:
Correctness, security, and fault tolerance
Dependable applications continue to work in the presence of
faults in hardware, software, and networks.

Presentation Outline

Introduction
Architectural Models
Software Layers
System Architectures

Client-Server

Alternative Client-Sever models driven by:

Clients and a Single Sever, Multiple Servers, Proxy Servers with

Caches, Peer Model
Mobile code, mobile agents, network computers, thin clients, mobile
devices and spontaneous networking

Design Challenges/Requirements

Fundamental Models formal description

Interaction, Failure, and Security models.
Summary

Lecture Overview (II)

Fundamental Models are concerned with a formal description of

the properties that are common in all of the architectural models.
All architectural models are composed of processes that
communicate with each other by sending messages over a
computer networks.
Models addressing time synchronization, message delays,
failures, security issues are:
Interaction Model deals with performance and the difficulty of
setting of time limits in a distributed system.
Failure Model specification of the faults that can be exhibited by
processes
Security Model discusses possible threats to processes and
communication channels.

Interaction Model

Computation occurs within processes;

The processes interact by passing messages,
resulting in:

Communication (information flow)

Coordination (synchronization and ordering of activities)
between processes.

Two significant factors affecting interacting

processes in a distributed system are:

Communication performance is often a limiting

characteristic.
It is impossible to maintain a single global notion of time.

Interaction Model:
Performance of Communication Channel

The communication channel in our model is realised in a variety

of ways in DSs. E.g., by implementation of:

Streams
Simple message passing over a network.

Communication over a computer network has performance

characteristics:

Latency:

Bandwidth:

A delay between the start of a messages transmission from one

process to the beginning of reception by another.
the total amount of information that can be transmitted over in a
given time.
Communication channels using the same network, have to share the
available bandwidth.

Jitter

The variation in the time taken to deliver a series of messages. It is

very relevant to multimedia data.

Interaction Model:
Computer clocks and timing events

Each computer in a DS has its own internal clock, which can be

used by local processes to obtain the value of the current time.
Therefore, two processes running on different computers can
associate timestamp with their events.
However, even if two processes read their clocks at the same
time, their local clocks may supply different time.
This is because computer clock drifts from perfect time and their
drift rates differ from one another.
Even if the clocks on all the computers in a DS are set to the
same time initially, their clocks would eventually vary quite
significantly unless corrections are applied.
There are several techniques to correct time on computer clocks.
For example, computers may use radio receivers to get readings
from GPS (Global Positioning System) with an accuracy about 1
microsecond.

Interaction Model:
Two variants of the interaction model

In a DS it is hard to set time limits on the time taken for process

execution, message delivery or clock drift.
Synchronous DS hard to achieve:
The time taken to execute a step of a process has known lower
and upper bounds.
Each message transmitted over a channel is received within a
known bounded time.
Each process has a local clock whose drift rate from real time
has known bound.
Asynchronous DS: There is NO bounds on:
Process execution speeds
Message transmission delays
Clock drift rates.

Interaction Model:
Event Ordering

In many DS applications we are interested in

knowing whether an event occurred before,
after, or concurrently with another event at
other processes.

The execution of a system can be described in

terms of events and their ordering despite the lack
of accurate clocks.

Consider a mailing list with:

users X, Y, Z, and A.

Real-time ordering of events

send
X

receive
m1
2
receive

receive

4
send
3

m2
receive

Physical
time

send

receive

receive
m3

A
t1

t2

m1

m2

receive receive receive

t3

Inbox of User A looks like:

Item
23

From
Z

Subject
Re: Meeting

24

Meeting

26

Re: Meeting

Due to independent delivery in message delivery, message may

be delivered in different order.
If messages m1, m2, m3 carry their time t1, t2, t3, then they can
be displayed to users accordingly to their time ordering.

Failure Model

In a DS, both processes and communication

channels may fail i.e., they may depart from
what is considered to be correct or desirable
behavior.
Types of failures:

Omission Failure
Arbitrary Failure
Timing Failure

Processes and channels

process p

process q

send m

receive

Communication channel
Outgoing message buffer

Incoming message buffer

Communication channel produces an omission failure if it

does not transport a message from ps outgoing
message buffer to qs incoming message buffer. This is
known as dropping messages and is generally caused
by a lack of buffer space at the receiver or at gateway or
by a network transmission error.

Omission and arbitrary failures

Classoffailure Affects
Failstop
Process

Description
Processhaltsandremainshalted.Otherprocessesmay
detectthisstate.
Crash
Process Processhaltsandremainshalted.Otherprocessesmay
notbeabletodetectthisstate.
Omission
Channel Amessageinsertedinanoutgoingmessagebuffernever
arrivesattheotherendsincomingmessagebuffer.
Sendomission Process Aprocesscompletesasend,butthemessageisnot
putinitsoutgoingmessagebuffer.
ReceiveomissionProcess Amessageisputinaprocesssincomingmessage
buffer,butthatprocessdoesnotreceiveit.
Arbitrary
Processor Process/channelexhibitsarbitrarybehaviour:itmay
(Byzantine)
channel send/transmitarbitrarymessagesatarbitrarytimes,
commitomissions;aprocessmaystoportakean
incorrectstep.

Timing failures

ClassofFailure Affects
Clock
Process
Performance

Process

Performance

Channel

Description
Processslocalclockexceedstheboundsonits
rateofdriftfromrealtime.
Processexceedstheboundsontheinterval
betweentwosteps.
Amessagestransmissiontakeslongerthanthe
statedbound.

Masking Failures

It is possible to construct reliable services from

components that exhibit failures.

For example, multiple servers that hold replicas of data can

continue to provide a service when one of them crashes.

A knowledge of failure characteristics of a

component can enable a new service to be designed
to mask the failure of the components on which it
depends:

Checksums are used to mask corrupted messages.

Security Model

The security of a DS can be achieved by

securing the processes and the channels
used in their interactions and by protecting
the objects that they encapsulate against
unauthorized access.

Protecting Objects: Objects and principals

Access rights

Object

invocation
Client
result

Principal (user)

Netw ork

Server

Principal (server)

Use access rights that define who is allowed to perform operation on a

object.
The server should verify the identity of the principal (user) behind each
operation and checking that they have sufficient access rights to perform
the requested operation on the particular object, rejecting those who do
not.

The enemy

Copy of m
The enemy
Processp

Processq
Communication channel

To model security threats, we postulate an enemy that is capable of

sending any process or reading/copying message between a pair of
processes

Threats form a potential enemy: threats to processes, threats to

communication channels, and denial of service.

Defeating security threats: Secure channels

PrincipalB

PrincipalA
Processp

Secure channel

Processq

Encryption and authentication are use to build secure channels.

Each of the processes knows the identity of the principal on
whose behalf the other process is executing and can check their
access rights before performing an operation.

Summary

Most DSs are arranged accordingly to one of a

variety of architectural models:

Client-Server

Alternative Client-Sever models driven by:

Clients and a Single Sever, Multiple Servers, Proxy Servers

with Cache, Peer Model
Mobile code, mobile agents, network computers, thin clients,
mobile devices and spontaneous networking

Fundamental Models formal description

Interaction, failure, and security models.
The concepts discussed in the module play an
important role while architecting DS and apps