Module -1

Distributed System Models and Enabling Technologies: Scalable Computing Over the Internet, Technologies for
Network Based Systems, System Models for Distributed and Cloud Computing, Software Environments for Distributed
Systems and Clouds, Performance, Security and Energy Efficiency.

Computer Technology Evolution:

Over the last 60 years, computing technology has changed a lot in areas like machine design, operating systems, network
connections, and how we use computers.

 Centralized vs. Distributed Computing:

Instead of relying on just one powerful computer, distributed computing uses many computers working together to solve large
problems over the Internet.

 Data-Intensive and Network-Centric:

Distributed computing systems focus more on handling large amounts of data and rely heavily on network connections.

 Modern Applications:
Many real-world applications today use parallel and distributed computing to solve problems.

 Impact on Society:
These large-scale Internet applications have greatly improved the quality of life and the availability of information services in
society today.

The Age of Internet Computing (Simplified)

 The Internet is used by billions of people: Every day, billions of people use the Internet for various activities.
 High-performance computing (HPC) services are needed: To handle so many people using the Internet, powerful
supercomputers and large data centers are needed to provide fast services to all the users at the same time.
 Old benchmarks aren't enough: In the past, the Linpack Benchmark was used to measure the performance of
supercomputers. However, it’s no longer the best way to measure performance because of the growing demand.
 Rise of cloud computing: Now, the focus has shifted to high-throughput computing (HTC), which involves using
many smaller, connected computers to perform tasks at the same time (parallel and distributed computing).
 Improving data centers: To meet the demand, we need to upgrade data centers with faster servers, better storage
systems, and faster networks.
 Goal: The main aim is to improve network-based computing and web services by using the latest technologies.

The Evolution of Computers (Simplified)

1. First Generation (1950-1970):

Mainframe computers were built for large businesses and government.
Examples: IBM 360, CDC 6400.
These computers were very big and expensive.
2. Second Generation (1960-1980):
Mini-computers became popular for smaller businesses and universities.
Examples: DEC PDP 11, VAX Series.
These computers were smaller and cheaper than mainframes.
3. Third Generation (1970-1990):
Personal computers (PCs) became common.
Built using very small and powerful microprocessors (VLSI).
PCs were now available for individual use in homes and small offices.
4. Fourth Generation (1980-2000):
Portable computers (laptops, notebooks) and small devices became widespread.
Many devices worked both with wired and wireless connections.
There was a huge increase in the number of personal computers and mobile gadgets.
Key Points:

 Computer technology evolves in generations.

 Each generation lasts 10-20 years, and some overlap.
 Early computers were large and expensive, but over time, they became smaller, more affordable, and portable.
 HPC and HTC Systems:
o HPC (High-Performance Computing) and HTC (High-Throughput Computing) systems have become more
common since 1990.
o These systems are used by both regular consumers and big web-scale companies for computing and data services.
 Trend of Shared Resources:
o The general trend in computing is to share web resources and use large amounts of data over the Internet.
o This makes computing more accessible and efficient.
 Evolution of HPC Systems:
o In the past, supercomputers (large and powerful machines) were the main form of HPC.
o Now, supercomputers are being replaced by clusters of cooperative computers. These clusters are groups of connected
computers that work together.
o The computers in a cluster are usually very similar to each other and are physically close together.
 HTC Systems and P2P Networks:
o HTC systems focus on processing many tasks over a longer period, rather than focusing on raw processing speed like
HPC.
o Peer-to-peer (P2P) networks are commonly used for tasks like file sharing and content delivery.
o P2P networks are made up of many client computers spread around the world. They don’t rely on one central machine.
 HTC and Cloud Computing:
o Cloud computing and web services are more related to HTC than HPC.
o These technologies focus on providing services to users over the Internet, using many distributed machines.
 Clustering and Grids:
o The combination of clustering and P2P technologies led to the creation of computational grids or data grids.
o These grids are systems that share computing power or data across a network of computers.

High-Performance Computing (HPC) Simplified

 HPC Growth Over Time:

o In the past, the focus of HPC systems was on increasing speed.
o The speed of these systems improved significantly from Gflops (billions of operations per second) in the early
1990s to Pflops (quadrillions of operations per second) by 2010.
o This increase in speed was mainly driven by the needs of scientific, engineering, and manufacturing fields.
 HPC Evolution:
o As computing evolved, several new technologies and systems emerged, like:
 Disparate nodes (different computing units working together).
 File sharing and distributed control.
 Geographically sparse systems (computers spread across different locations).
 Virtualization (running multiple systems on a single machine).
 Clusters and MPP (Massively Parallel Processing) systems.
 P2P (Peer-to-Peer) networks.
 Cloud computing and the Internet of Things (connected devices like sensors and RFID).
 Web 2.0 services and Service-Oriented Architecture (SOA).
 Supercomputers:
o The Top 500 supercomputers are ranked based on their speed using a benchmark called Linpack.
o These supercomputers are used mainly by less than 10% of all computer users.
o The majority of computer users today use desktop computers or large servers for tasks like internet
searches and market-driven computing.

High-Throughput Computing (HTC) Explained:

1. Shift from HPC to HTC:

o The focus in computing is changing from High-Performance Computing (HPC) to High-Throughput
Computing (HTC).
o In HPC, the goal is to perform a few tasks really quickly, while in HTC, the focus is on completing many
tasks efficiently over time.
2. Applications of HTC:
o HTC is mainly used for applications that handle a large number of tasks at once, such as:
 Internet searches (like Google).
 Web services that serve millions or more users at the same time.
3. Performance Goals in HTC:
o Instead of focusing on how fast a single task is completed (like in HPC), HTC measures how many tasks can
be completed in a certain time period. This is called high throughput.
4. Improving HTC:
o HTC systems aim to improve:
 Batch processing speed: Handling many tasks at once.
 Cost efficiency: Reducing the cost of running large-scale operations.
 Energy savings: Making systems more energy-efficient.
 Security and reliability: Ensuring data is safe and the system is stable.
5. Purpose of the Book:
o The book will cover both HPC and HTC systems to help users understand how to meet the needs of different
computing tasks, from high-speed calculations to handling large amounts of data efficiently.

Overview:

 P2P Networks & Clusters: Since the mid-1990s, technologies have been developed to build large networks for
computing, like P2P (Peer-to-Peer) networks and clusters (groups of computers working together).
 Computational Grids and Data Grids: These networks have evolved into large projects known as computational
grids and data grids, designed to share resources like hardware, software, and data.
 Cloud Computing: Recently, there’s been more focus on using cloud resources for data-heavy tasks. Cloud
computing involves using server clusters and large databases stored in data centers, instead of personal desktop computers.

Key Concepts of Distributed Systems:

 Grid & Cloud Systems: These systems focus on resource sharing across hardware, software, and data. The goal is to
maximize parallelism (doing many things at once) across multiple machines.

Examples of High-Performance Systems:

 Tianhe-1A: In October 2010, a powerful cluster machine was built in China with over 86,000 CPU cores and over 3
million GPU cores.
 Computational Grids: Some grids connect hundreds of server clusters and involve thousands of machines.
 P2P Networks: Some Peer-to-Peer (P2P) networks have millions of machines working at once.

Cloud Computing:

 Experimental Clusters: Large-scale cloud computing clusters have been built with thousands of processing nodes.
 Chapters 4-6 focus on cloud computing and how it works in systems like data centers, social networks, and
virtualized platforms.

Future of High-Performance Computing (HPC) and High-Throughput Computing (HTC):

 Multicore Processors: Both HPC (used for science and engineering) and HTC (used for business) systems will need
multicore or many-core processors to handle many computing threads (tasks) at once.
 Parallelism: Both systems focus on running tasks in parallel and distributed computing across many machines.

Design Objectives:

To meet future demands, computing systems must focus on the following goals:

Efficiency:

o In HPC: It means using resources effectively by running many tasks at the same time.
o In HTC: It focuses on handling lots of jobs, accessing data quickly, and using energy efficiently.
2. Dependability:
o Systems must be reliable, meaning they can keep running even when parts of the system fail.
o They must provide high-throughput service with guaranteed Quality of Service (QoS), even during failures.
3. Adaptation:
o The programming model must be flexible enough to handle billions of job requests on massive datasets,
especially with cloud resources that can change in workload.
4. Flexibility in Application Deployment:
o Distributed systems need to run efficiently in both HPC applications (like science/engineering) and HTC
applications (like business).
echnologies for Network-Based Systems

In this section, we’ll explore how hardware, software, and network technologies are used in distributed computing systems.
We'll focus on how distributed operating systems handle large-scale computing, especially using parallel processing.

1.2.1 Multicore CPUs and Multithreading Technologies

Over the past 30 years, computer technology has rapidly improved. These improvements in processors and networks are key to
building High-Performance Computing (HPC) and High-Throughput Computing (HTC) systems.

In this section, we’ll look at:

 Processor speed: How fast processors can perform tasks (measured in millions of instructions per second, MIPS).
 Network bandwidth: The amount of data a network can handle (measured in megabits per second, Mbps, or gigabits
per second, Gbps).

1.2.1.1 Advances in CPU Processors

 Multicore Processors: Modern processors have multiple cores (e.g., dual, quad, or more), allowing them to perform
many tasks simultaneously.
 Parallelism: These cores help exploit two kinds of parallelism:
o ILP (Instruction-Level Parallelism): Running multiple instructions in one clock cycle.
o TLP (Thread-Level Parallelism): Running multiple threads of a program at the same time.

 Processor Speed Over Time:

o In 1978, processors like the VAX 780 could handle 1 MIPS (Million Instructions Per Second).
o By 2002, the Intel Pentium 4 had 1,800 MIPS.
o By 2008, the Sun Niagara 2 reached 22,000 MIPS.

 Moore’s Law: This law, which says the number of transistors on a chip doubles about every two years, has proven
accurate, leading to better performance.

Processor Clock Rate Limits

 Clock Rate Growth: Processor speed also improved over time, from 10 MHz in older chips like Intel 286 to 4 GHz in
newer ones like Pentium 4.
 Challenges with High Clock Rates:
o Heat Generation: When processors run at very high speeds, they generate a lot of heat, which limits how
fast they can go.
o Power Consumption: High clock speeds also require more power, making it hard to increase speeds much
further.

 Current Status: Few processors exceed 5 GHz today, meaning there are limits to how much clock speeds can grow
unless chip technology improves.

Techniques for Maximizing Processor Efficiency

 ILP Techniques: Modern CPUs use techniques like:

o Superscalar Architecture: Running multiple instructions at the same time.
o Dynamic Branch Prediction: Predicting the next instruction to reduce delays.
o Speculative Execution: Executing instructions before they’re confirmed to increase speed.
  Support for ILP: These techniques need special support from both hardware (the physical chip) and software (the
programs that run on it).
 DLP and TLP in GPUs: Graphics Processing Units (GPUs) use many simple cores to handle parallel tasks like
processing images or video. GPUs often have hundreds or even thousands of cores that handle tasks in parallel.

Distributed and Cloud Computing Systems

1. What are Distributed and Cloud Computing Systems?

o These systems are made up of many independent computers, or "nodes," connected together to work as a
single system.
o The computers are connected using different types of networks like:
 SAN (Storage Area Networks)
 LAN (Local Area Networks)
 WAN (Wide Area Networks)

The connection can be in a hierarchical manner, meaning they can be organized in a structured way.

2. How do these systems work?

o Modern networking technology allows many computers to be connected easily. For example:
 A few LAN switches can connect hundreds of computers to form a working cluster.
 WANs (Wide Area Networks) can connect many of these local clusters, forming a huge system.

These networks can create massive systems with millions of computers, connected through edge networks.

3. What is a "Massive System"?

o A massive system is one that is highly scalable, meaning it can grow easily in size. It can connect many
computers and expand to handle large-scale needs like web-scale connectivity (think about large websites like Google,
Facebook, etc.).
4. Types of Massive Systems (Table 1.2)
o There are four main types of large-scale systems:

1. Clusters: Groups of computers connected together to work as a single unit.

2. P2P (Peer-to-Peer) Networks: Computers communicate directly with each other without a central server.
3. Computing Grids: A network of computers that share resources to solve complex tasks.
4. Internet Clouds (Data Centers): Large groups of computers that provide cloud computing services.
5. Scale of these Systems:

These systems can involve hundreds, thousands, or even millions of computers.The computers in these systems work together in
different ways:

 Collectively: All work together on the same task.

 Cooperatively: Work together but may have different roles.
 Collaboratively: Work together on different parts of a larger task.

This breakdown highlights the basics of distributed and cloud systems, helping you understand how they function and scale up
in the modern world of computing.

 Clusters in Supercomputing:

 In 2009, 417 of the world's top 500 supercomputers used cluster architecture (group of connected computers
working together).
 Clusters are important for creating large-scale grids and cloud systems.

 P2P Networks for Business:

 P2P (Peer-to-Peer) networks are mostly used in business applications.

 However, content industries (like music, movies) were hesitant to use P2P due to concerns about copyright
protection.

 Challenges for National Grids:

 Many national grids (large networks of computers) built in the past were not used effectively.
 This was mainly because of poor middleware (software tools to manage grids) and badly written applications.

 Advantages of Cloud Computing:

 Cloud computing is becoming popular because it's low-cost and simple for both providers and users.

 Server Cluster Architecture:

 A server cluster is a group of connected computers (called nodes) working together.

 These clusters use a network to connect the nodes. The network can be:
o SAN (Storage Area Network) like Myrinet.
o LAN (Local Area Network) like Ethernet.

 Building Larger Clusters:

 For larger clusters with more nodes, the network can include:
o Multiple Gigabit Ethernet switches.
o Myrinet or InfiniBand switches.
 The network setup can be designed to handle more nodes by organizing the switches in levels.

 Scalable Clusters:

 Using a SAN, LAN, or WAN, you can scale the cluster to include many nodes, increasing its power and capacity.

 Internet Connectivity:

 The cluster is connected to the Internet through a VPN (Virtual Private Network) gateway.
 The VPN gateway helps locate the cluster by providing an IP address.

 System Image:

 A system image refers to how the operating system (OS) manages the cluster's resources.
 In most clusters, each node has its own OS, so they work independently.
 This means that each node has its own system image and runs its own OS, making the cluster have multiple system
images.

 Distributed and Cloud Computing Systems:

 These systems are made up of many independent computers (nodes) that are connected together.
 These computers can be linked by networks like SANs, LANs, or WANs in a structured way.
 A LAN (Local Area Network) connects a smaller group of machines, while a WAN (Wide Area Network) can
connect multiple LANs to create a larger network.

 Scaling Up:

 Using modern networking technology, it's easy to connect hundreds of machines using a few switches in a local
cluster.
 WANs can link many of these local clusters, creating a massive network of machines that can be spread across large
areas.

 Massive Systems:

 These systems are called highly scalable, meaning they can easily grow in size and handle a large number of
machines.
 They can reach web-scale connectivity, which means they can support millions of connected devices, either physically
or logically.

 Four Types of Large Systems (from the table):

1. Clusters: Groups of connected computers that work together as one system.

 2. P2P Networks (Peer-to-Peer): A decentralized network where computers connect directly to each other.
3. Computing Grids: Systems that allow distributed computing power across many machines.
4. Internet Clouds: Large-scale systems using many data centers to provide services over the internet.

 Scale of these Systems:

 These systems can involve hundreds, thousands, or even millions of computers working together.
 The machines in these systems can work together in different ways—either collectively, cooperatively, or
collaboratively—to achieve a goal.

SOFTWARE ENVIRONMENTS FOR DISTRIBUTED SYSTEMS AND CLOUDS

Service-Oriented Architecture (SOA)

 Basic Concept of Entities:

 In grid systems/web services, an entity is a service (a unit of work or task).

 In Java, an entity is a Java object (a piece of data or code that performs tasks).
 In CORBA, an entity is a CORBA distributed object (an object that can communicate over a network in many
different languages).

 OSI Layers:

 All these technologies are built on the OSI model, which is a seven-layer system that helps computers communicate
with each other over a network.
 The OSI model gives basic networking rules, like how to send data, how to address computers, and how to handle
errors.

 Base Software Environments:

 On top of the OSI layers, there is a base software environment that handles tasks like communication and execution
of code:
o For web services, it could be something like .NET or Apache Axis.
o For Java, it is the Java Virtual Machine (JVM).
o For CORBA, it uses a broker network to manage communication between objects.

 Higher-Level Features:

 Above the base software, there is a higher-level environment that makes it easier to work with distributed systems.
 This higher level focuses on how entities communicate with each other and the interfaces through which they
interact. This is like how applications or services talk to each other across networks.

 Entity Communication:

 This communication system rebuilds the top four layers of the OSI model, but instead of working with bits (the
lowest level of data), it works at the level of entities (services, objects, or distributed objects).

The system helps distributed systems (like web services, Java, and CORBA) organize how entities work together and
communicate, using a layered approach on top of the basic OSI networking model.