✅ What is a Graph?

Definition:

 A graph is a data structure consisting of:

o Vertices (nodes): represent entities.
o Edges (links): represent relationships between vertices.

Types:

1. Undirected Graph:
o Edges do not have direction.
o If there is an edge between A and B, you can go from A → B and B → A.
2. Directed Graph (Digraph):
o Edges have direction.
o A → B does not imply B → A.
3. Weighted Graph:
o Each edge has a weight (cost, distance, etc.).
4. Unweighted Graph:
o Edges have no weights.
5. Cyclic Graph:
o Contains one or more cycles.
6. Acyclic Graph:
o Contains no cycles.
o Directed Acyclic Graph (DAG) used in scheduling, compiler design, etc.
7. Connected Graph:
o There is a path between every pair of vertices.
8. Disconnected Graph:
o Not all vertices are connected.

🔍 Breadth-First Search (BFS)

Algorithm:

 Explores all neighbors at the current depth before moving to the next level.

Steps:

1. Start at a source node.

2. Use a queue to explore neighbors level-wise.
3. Mark visited nodes to avoid cycles.

Applications:

 Finding shortest path in unweighted graphs.

 Social networks (finding people within 2-3 connections).
  Web crawling.
 GPS navigation.

Advantages:

 Guarantees shortest path in unweighted graphs.

 Simple and easy to implement.

Disadvantages:

 Requires more memory due to queue.

 Can be slow for wide/deep graphs.

🔎 Depth-First Search (DFS)

Algorithm:

 Explores as far as possible along each branch before backtracking.

Steps:

1. Start at a node.
2. Use a stack (or recursion) to go deep.
3. Backtrack if no unvisited neighbors.

Applications:

 Detecting cycles in graphs.

 Topological sorting.
 Maze solving.

Advantages:

 Requires less memory than BFS.

 Better for exploring deeper structures.

Disadvantages:

 May not find the shortest path.

 Can get trapped in cycles (if not tracked).

� Dijkstra’s Shortest Path Algorithm

Purpose:
  Finds the shortest path from a source node to all other nodes in a weighted graph.

Key Points:

 Works with non-negative weights.

 Uses a priority queue (min-heap).

Steps:

1. Initialize all distances as ∞, source as 0.

2. Pick the node with the smallest distance.
3. Update neighbors’ distances if a shorter path is found.
4. Repeat until all nodes are visited.

Applications:

 GPS systems.
 Network routing.
 Game AI pathfinding.

🌲 Minimum Spanning Tree (MST)

Definition:

 A subset of edges that connect all vertices with the minimum total edge weight and
no cycles.

Applications:

 Network design.
 Clustering.
 Image segmentation.

🔧 Prim’s Algorithm
Purpose:

 Builds the MST by growing the tree from one starting node.

Steps:

1. Start from any node.

2. Add the smallest edge that connects a new node to the MST.
3. Repeat until all nodes are included.
Differences from Kruskal’s:

 Prim’s grows the MST one node at a time.

 Kruskal’s adds edges in increasing order.

🔍 Informed Search Algorithms

Definition:

 Search algorithms that use heuristics (estimates) to make intelligent decisions.

Key Types:

1. Best-First Search:
o Uses a priority queue based on a heuristic value.
o Greedy in nature.
2. A Search*:
o Combines actual cost from start and estimated cost to goal:
f(n) = g(n) + h(n)
 g(n): cost so far
 h(n): estimated cost to goal

Advantages:

 Efficient in finding optimal solutions.

 Reduces search space.

Disadvantages:

 Depends heavily on the quality of the heuristic.

 Can be memory-intensive.

� Pure Heuristic Search

Definition:

 Search purely based on the heuristic function h(n).

Example: Greedy Best-First Search

 Chooses node with the lowest h(n) regardless of cost so far.

Advantage:
  Fast and often finds solution quickly.

Disadvantage:

 Not guaranteed to find the shortest path.

⭐ A* Search Algorithm
Key Features:

 Combines actual cost (g) and heuristic (h).

 Guarantees optimal and complete results if h(n) is admissible (never overestimates).

Applications:

 Pathfinding in games.
 Robotics navigation.
 Puzzle solvers.

Advantages:

 Optimal and efficient.

 Adapts to various environments.

Disadvantages:

 High memory usage.

 Requires a good heuristic.

🔎 Linear Search Algorithm

Definition:

 Sequentially checks each element in a list.

Steps:

1. Start at index 0.
2. Compare each element with the target.
3. Return index if found, else return -1.

Best Case: First element match → O(1)

Worst Case: Last/no match → O(n)
Advantages:

 Simple to implement.
 Works on unsorted data.

Disadvantages:

 Slow for large datasets.

🔍 Parallel Search Algorithm

Definition:

 Searches a data structure by splitting the workload across multiple processors.

Types:

 Parallel Linear Search.

 Parallel Binary Search.
 Divide-and-conquer approaches.

Advantages:

 Speeds up searching in large datasets.

 Utilizes multi-core CPUs and GPUs.

Disadvantages:

 Overhead of synchronization.
 Complex to implement for irregular data.

🏗� What is Parallel Architecture?

Definition:

 Hardware systems that support multiple computations simultaneously.

Types of Parallel Architectures:

1. SISD (Single Instruction, Single Data):

o Traditional single-core system.
2. SIMD (Single Instruction, Multiple Data):
o Same operation on multiple data (e.g., GPUs).
3. MISD (Multiple Instruction, Single Data):
 o
Rare; multiple operations on the same data stream.
4. MIMD (Multiple Instruction, Multiple Data):
o Multi-core processors and distributed systems.

⚙� Parallel Programming Issues

1. Synchronization:

 Ensuring correct execution order between threads.

2. Deadlock:

 Processes wait on each other indefinitely.

3. Race Conditions:

 Output depends on the timing of threads.

4. Load Balancing:

 Equal distribution of workload.

5. Scalability:

 Ability to handle increased processors/data.

6. Communication Overhead:

 Time consumed in data exchange between threads/nodes.

💬 What is MPI?
MPI (Message Passing Interface):

 A standardized, portable message-passing system for parallel computing.

 Used in distributed memory systems.

🎯 Goals of MPI:

1. Portability across platforms.

 2. Efficiency and scalability.
3. Support for point-to-point and collective communication.
4. Enable parallelism in C, C++, and Fortran.

👋 Hello World Program in MPI

🔹 MPI in C:
c
CopyEdit
#include <mpi.h>
#include <stdio.h>

int main(int argc, char** argv) {

MPI_Init(&argc, &argv);

int rank, size;

MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &size);

printf("Hello world from process %d of %d\n", rank, size);

MPI_Finalize();
return 0;
}

🔹 MPI in FORTRAN:
fortran
CopyEdit
program hello
use mpi
implicit none
integer :: rank, size, ierr

call MPI_Init(ierr)
call MPI_Comm_rank(MPI_COMM_WORLD, rank, ierr)
call MPI_Comm_size(MPI_COMM_WORLD, size, ierr)

print *, 'Hello world from process', rank, 'of', size

call MPI_Finalize(ierr)
end program hello

🔄 Blocking vs Nonblocking Communication

Feature Blocking Nonblocking
Communication Sender/receiver waits Continues after call
Control Return After message is sent/received Immediately
Example Function MPI_Send, MPI_Recv MPI_Isend, MPI_Irecv
 Feature Blocking Nonblocking
Performance Easier but slower Faster with overlap potential
Use Case Simple logic Complex, high-performance apps

� MPI Program Structure

1. Initialization:
o MPI_Init()
2. Get process info:
o MPI_Comm_rank(), MPI_Comm_size()
3. Communication:
o Send/Receive or Broadcast data.
4. Finalization:
o MPI_Finalize()

🗃� MPI Data Types

Standard Types:

 MPI_INT, MPI_FLOAT, MPI_DOUBLE, MPI_CHAR

Derived Types:

 Create custom data layouts using:

o MPI_Type_contiguous
o MPI_Type_vector
o MPI_Type_struct

Use Case: Sending structs or arrays efficiently.

🌐 What is Virtual Topology?

Definition:

 A logical arrangement of processes independent of physical hardware.

🔄 MPI Virtual Topology Routines:

 MPI_Cart_create: Creates a Cartesian topology.

 MPI_Cart_coords: Finds coordinates of a process.
 MPI_Cart_shift: Finds neighbors in a grid.
 MPI_Graph_create: Creates arbitrary topologies.
Types:

 Cartesian: Grid-like arrangement.

 Graph: Custom layout.

📡 Eager Protocol
Definition:

 Sends data immediately without waiting for the receiver’s readiness.

Use Case:

 Small messages.
 Reduces latency.

Advantage:

 Fast transmission.

Disadvantage:

 Risk of buffer overflow.

🔁 Rendezvous Protocol
Definition:

 Sender waits until receiver is ready.

Use Case:

 Large messages.

Advantage:

 Safe; prevents buffer overflow.

Disadvantage:

 Higher latency.
🔃 Linear Shift Communication Pattern
Definition:

 Each process sends data to its neighbor in one direction (left/right).

Steps:

1. Process i sends to i+1 and receives from i-1.

2. Wraps around in circular structures.

Use Case:

 Ring topology.
 Collective operations.

🚫 Avoiding Implicit Serialization

Problem:

 Sequential execution due to poor synchronization.

Solutions:

 Use non-blocking calls.

 Pipeline communication.
 Proper ordering using MPI_Barrier.

Goal:

 Maximize parallel overlap and performance.

🌐 What is Network Contention?

Definition:

 When multiple data transmissions compete for the same network resources.

Causes:

 Many-to-one communication.
 Shared interconnects.

Effects:
  Increased latency.
 Lower bandwidth.

Solutions:

 Topology-aware scheduling.
 Buffer management.
 Traffic shaping.

📘 Dynamic Programming (DP)

Definition:

 A method to solve complex problems by breaking them down into overlapping

subproblems, solving each subproblem once, and storing its result.

Key Concepts:

1. Optimal Substructure:
o Problem can be solved using solutions of its subproblems.
2. Overlapping Subproblems:
o Same subproblems are solved multiple times.

Techniques:

 Top-Down (with memoization)

 Bottom-Up (tabulation)

Examples:

 Fibonacci series.
 Knapsack problem.
 Matrix chain multiplication.
 Shortest paths in graphs (e.g., Floyd-Warshall).

Advantages:

 Efficient for problems with overlapping subproblems.

 Avoids redundant computation.

Disadvantages:

 High memory usage due to table storage.

🛣� Shortest Path Problem
Definition:

 Finding the path between two vertices such that the sum of weights is minimized.

Algorithms:

1. Dijkstra’s Algorithm:
o For non-negative weights.
2. Bellman-Ford Algorithm:
o Handles negative weights.
3. Floyd-Warshall Algorithm:
o Finds shortest paths between all pairs of vertices.
4. A*:
o Uses heuristic for efficient search.

Applications:

 GPS and route planning.

 Network routing.
 Robotics and game AI.

� What is Neural Network?

Definition:

 A neural network is a computational model inspired by the human brain.

 Made up of neurons (nodes) organized in layers:
o Input layer
o Hidden layers
o Output layer

Evolution of Neural Networks:

1. Perceptron (1958): Single-layer neural network.

2. Multi-Layer Perceptron (MLP): Introduced hidden layers.
3. Backpropagation: Enabled training of deeper networks.
4. CNN (1998): Efficient for image data.
5. RNN: Designed for sequential data.
6. Deep Learning: Modern NN with many hidden layers.

� Types of Neural Networks

 1. Feedforward Neural Network (FNN):
o Data flows one way.
o Used for classification/regression.
2. Convolutional Neural Network (CNN):
o Specialized for image processing.
o Uses convolution and pooling layers.
3. Recurrent Neural Network (RNN):
o Has loops to handle sequences.
o Suitable for time series, NLP.
4. Radial Basis Function (RBF) Network:
o Uses radial basis functions as activation.
o Fast learning but sensitive to noise.
5. Generative Adversarial Networks (GANs):
o Generator and discriminator networks.
o Used for image generation.

� Convolutional Neural Network (CNN)

Definition:

 A class of deep neural networks mainly used for image recognition and
classification.

🔧 Architecture:

1. Input Layer: Image pixels.

2. Convolutional Layer: Detects features using filters.
3. Activation (ReLU): Introduces non-linearity.
4. Pooling Layer: Downsamples feature maps.
5. Fully Connected Layer: Final classification.
6. Output Layer: Class probabilities.

📈 Advantages:

 Reduces number of parameters.

 Captures spatial hierarchy.
 High accuracy in vision tasks.

📉 Disadvantages:

 Computationally expensive.
 Requires a lot of labeled data.

🔁 What is RNN?
Definition:

 A neural network where connections form cycles, allowing it to remember previous

outputs.

🔄 How it works:

 Each time step takes input and previous hidden state.

 Output depends on current input and memory.

Formula:
h(t) = f(W * x(t) + U * h(t-1))

🔄 RNN – Advantages & Disadvantages

✅ Advantages:

 Remembers sequences.
 Ideal for language modeling, time series, speech recognition.

❌ Disadvantages:

 Vanishing gradient problem.

 Hard to learn long-term dependencies.
 Training is slow.

📚 Applications of Neural Networks

1. Image Recognition
2. Speech Recognition
3. Natural Language Processing
4. Medical Diagnosis
5. Fraud Detection
6. Autonomous Vehicles
7. Recommendation Systems

🏥 Neural Networks in Healthcare

Use Cases:

 Diagnosing diseases (e.g., cancer detection).

  Predicting patient outcomes.
 Analyzing medical images (MRI, CT).
 Drug discovery.
 Robotic surgeries.

Benefits:

 Speed, accuracy, and real-time decision-making.

💰 Financial Applications of Neural Networks

1. Credit Scoring:
o Analyze risk based on transaction data.
2. Stock Price Prediction:
o Predict market trends using time series models (RNN/LSTM).
3. Fraud Detection:
o Detect unusual patterns in transactions.
4. Algorithmic Trading:
o Make high-speed trading decisions.
5. Loan Default Prediction:
o Assess borrowers' risk using neural patterns.