Deep Learning Unit-4

1) Explain RNN with its types. [6 Marks]

RNN (Recurrent Neural Network) is a type of neural network used for processing sequential data like time series, speech, or text. Unlike
regular neural networks, RNNs have loops that allow information to be passed from one step to the next, making them good at
remembering previous inputs.

Types of RNN:

1. Basic RNN: Standard type that uses previous hidden state and current input to produce output. It struggles with long-term memory.
2. LSTM (Long Short-Term Memory): Solves the short-term memory problem. It uses gates (input, forget, output) to control what to
remember or forget.
3. GRU (Gated Recurrent Unit): Similar to LSTM but has fewer gates (reset and update). It’s simpler and faster than LSTM.
4. Bidirectional RNN: Processes input in both forward and backward directions, capturing more context.
5. Deep RNN: Uses multiple RNN layers to learn complex features

2) Explain in brief Encoder-Decoder Architecture. [6 Marks]

Encoder-Decoder Architecture is used in tasks like machine translation (e.g., English to French). It works in two steps:

1. Encoder: Takes input sequence (like a sentence) and converts it into a fixed-length context vector. It summarizes the input.
2. Decoder: Uses the context vector to generate the output sequence (translated sentence or response).

This architecture is commonly used in: Language translation, Chatbots, Text summarization

Advanced versions use attention mechanism to focus on important parts of the input during decoding.

2) Explain Different types of Deep Learning. [6 Marks]

There are several types of Deep Learning models used for different tasks:

1. Feedforward Neural Network (FNN): Basic type where information flows in one direction. Used for simple prediction tasks.
2. Convolutional Neural Network (CNN): Best for image processing, like face or object recognition. It detects patterns using filters.
3. Recurrent Neural Network (RNN): Best for sequential data like speech, text, or time series. It remembers past inputs.
4. Autoencoders: Used for data compression and noise removal. Learns to compress and then reconstruct input data.
5. Generative Adversarial Networks (GANs): Used to generate new data like fake images. Works with a Generator and
Discriminator competing with each other.
6. Transformer: Modern model for language tasks like translation or summarization. Uses attention instead of recurrence, and it's
very fast and accurate.

4) WSN on Performance Matrices [6 Marks]

WSN (Wireless Sensor Network) performance is evaluated using several performance metrics (or matrices) that measure efficiency
and effectiveness:

1. Energy Efficiency: Since sensor nodes run on batteries, energy consumption must be minimal to increase network lifetime.
2. Latency: Time taken for data to travel from sensor node to base station. Lower latency is better.
3. Throughput: Amount of data successfully transmitted per unit time.
4. Packet Delivery Ratio (PDR): Ratio of successfully delivered packets to the total packets sent.
5. Scalability: The network’s ability to maintain performance as the number of nodes increases.
6. Coverage: Area monitored effectively by the sensor network.
7. Fault Tolerance: Ability of the network to continue functioning despite node failures.

These matrices help in evaluating and improving the performance of WSN applications like environmental monitoring, health tracking, etc.

5) Compare Implicit and Explicit Memory [6 Marks]

Feature Implicit Memory Explicit Memory

Definition Memory we use unconsciously Memory we recall consciously
Also Known As Non-declarative memory Declarative memory
Examples Riding a bicycle, typing, habits Remembering a fact, a name, or a date
Type of Knowledge "How to" knowledge "What is" knowledge
Brain Area Involved Basal ganglia and cerebellum Hippocampus and frontal lobes
Conscious Awareness Not required Required

6) What are Default Baseline Models? Explain in Brief. [6 Marks]

Default baseline models are simple models used to set a reference point for evaluating the performance of more complex machine
learning models.
Purpose:

 They help determine whether a new model actually adds value.

 If your advanced model does not beat the baseline, it's not useful.

Examples:

1. Mean or Median Predictor: For regression, predict the average value.

2. Most Frequent Class: For classification, always predict the most common label.
3. Random Guess: Predict randomly to compare with intelligent models.

Baseline models are useful in early stages of model development or when testing the effectiveness of algorithms.

7) Explain Recurrent Neural Network (RNN) with its Architecture [6 Marks]

A Recurrent Neural Network (RNN) is a neural network designed to process sequential data by maintaining a hidden state that
captures past information.

RNN Architecture:

 Input Layer: Takes input sequence (e.g., words or time-series data).

 Hidden Layer: The core of RNN. It receives current input and previous hidden state to produce a new hidden state.
 Output Layer: Generates output at each time step (optional depending on task).

Key Features:

 Shared weights: The same weights are used at every time step.
 Feedback Loop: Hidden state carries information from past steps (temporal memory).

Uses: Language modeling, speech recognition, time-series prediction.

8) Draw and Explain Architecture for Long Short-Term Memory (LSTM) [6 Marks]

LSTM Architecture Diagram: Explanation:

LSTM is a type of RNN designed to solve the vanishing gradient problem by introducing a memory cell and gates to control the flow of
information.

1. Forget Gate: Decides what to forget from the cell state.\

2. Input Gate: Decides what new information to store.

3. Cell State Update:

4. Output Gate: Controls what to output.

9) Explain How the Memory Cell in LSTM is Implemented Computationally [6 Marks]

 The LSTM memory cell is implemented using gates and activation functions to manage long-term dependencies.

Computational Implementation:

1. Vectors and Matrices: Each gate (input, forget, output) uses its own weight matrices and bias vectors.
2. Element-wise Operations: Gates use sigmoid activation (output between 0 and 1) to decide "how much" to keep or discard.
3. Memory Cell Update:
o Forget Gate: Multiplies old cell state to forget part of it.
o Input Gate: Adds new candidate information to cell state.
o Output Gate: Produces final output from updated cell state.

Key Operations:

 Matrix multiplications (W⋅x+U⋅h)

 Activation functions: sigmoid (σ), tanh
 Element-wise addition and multiplication

This allows LSTM to retain or discard information across long sequences, making it better at tasks like language translation or time-
series forecasting.

10) Explain Deep Generative Model with Example [6 Marks]

A Deep Generative Model is a type of deep learning model that can generate new data that looks similar to the training data. It learns
the probability distribution of the input data and uses it to produce similar outputs.

Key Idea: Instead of just classifying or predicting, generative models create new samples like images, text, or audio.

Types:

 Variational Autoencoders (VAEs)

 Generative Adversarial Networks (GANs)

Example: A GAN trained on human face images can generate new realistic face images that do not belong to any real person.

Use Cases:

 Data augmentation
 Image synthesis
 Text generation

11) How does GAN Training Scale with Batch Size? [6 Marks]

In GAN (Generative Adversarial Network) training, batch size affects both training stability and output quality.

Impact of Batch Size:

1. Small Batch Size:

o More noisy gradients
o Better generalization
o Slower convergence
o May lead to mode collapse
2. Large Batch Size:
o Smoother gradient updates
o Faster training
o Requires more memory (GPU/TPU)
o May lead to less diversity in generated images

Scaling Issues:

 Too large batch size can make training unstable or make the Discriminator overpower the Generator.
 Some architectures like StyleGAN use progressive growing and adaptive batch sizes to maintain stability.

Conclusion: Batch size must be carefully selected based on hardware and model behavior.

12) List the Applications of GAN Network with Description [6 Marks]

Here are important applications of GANs with brief descriptions:

 1. Image Generation:
o GANs can create high-quality, realistic images from noise.
o Example: Generating fake celebrity faces (ThisPersonDoesNotExist.com).
2. Image-to-Image Translation:
o Convert one image type to another.
o Example: Convert black & white to color, sketch to photo.
3. Data Augmentation:
o Generate extra training data to improve ML models.
o Useful when real data is limited (e.g., medical images).
4. Super-Resolution:
o Improve the resolution of low-quality images.
o Example: Enhance blurred CCTV footage.
5. Text-to-Image Generation:
o GANs generate images from text descriptions.
o Example: “A cat sitting on a red sofa” → generates matching image.
6. Art and Music Creation:
o GANs create original art pieces, music, and designs.
7. Deepfake Generation:
o Generate fake but realistic videos or audio of people (used in both creative and malicious ways).

13) What is RNN? What is the need of RNN? Explain working of RNN. [6 Marks]

RNN (Recurrent Neural Network) is a type of neural network designed to process sequential data by using loops in its architecture to retain
memory of previous inputs.

Need of RNN:Traditional neural networks treat each input independently. But in tasks like speech recognition, language modeling, or
time-series prediction, past information is important. RNNs remember previous inputs, making them suitable for such tasks.

Working of RNN: At each time step t, RNN takes:

o Current input xt
o Previous hidden state ht−1

 It calculates the current hidden state ht as:

 The output is generated based on the hidden state.
 It processes input step-by-step, sharing weights across time.

14) How LSTM and Bidirectional LSTM Work [6 Marks]

LSTM (Long Short-Term Memory):

LSTM is a special kind of RNN that solves the vanishing gradient problem by introducing a cell state and gates.

 Forget Gate: Decides what to remove from memory.

 Input Gate: Adds new data to memory.
 Output Gate: Decides what to output.

These gates allow LSTM to remember long-term dependencies.

Bidirectional LSTM:

 It uses two LSTMs: one processes the sequence forward, the other backward.
 Their outputs are combined (e.g., concatenated).
 This helps the model get context from both past and future inputs.

Use Case: Very useful in NLP tasks like named entity recognition or machine translation.

15) Explain Unfolding Computational Graphs with Example [5 Marks]

Unfolding means breaking down a recurrent network over time into a sequence of operations (like a feedforward network).

Example:

An RNN running over 3 time steps is "unfolded" into:

Each time step is treated like a layer in the graph:

x1 → h1 → x2 → h2 → x3 → h3

This unfolded graph makes it easier to apply backpropagation through time (BPTT) to update weights during training.

16) What are types of RNN? How to train RNN? [6 Marks]

Types of RNN:

1. Vanilla RNN:
o Basic RNN model with one hidden state looped over time.
o Prone to vanishing gradient problems.
2. LSTM (Long Short-Term Memory):
o Designed to remember long-term dependencies using gates and memory cells.
3. GRU (Gated Recurrent Unit):
o A simpler version of LSTM with fewer gates (no separate cell state).
4. Bidirectional RNN:
o Processes data in both forward and backward directions.
5. Deep RNN:
o Has multiple RNN layers stacked on top of each other for deeper learning.

How to Train RNN:

1. Input Sequences are passed through the RNN.

2. At each time step, a hidden state is updated using the input and previous state.
3. Output is generated at each step or at the end (depending on task).
4. Use Backpropagation Through Time (BPTT):
o Unfold the RNN across time steps.
o Apply gradient descent and update weights using the loss.
5. Use optimizers like Adam or SGD.

17) Explain Encoder-Decoder Sequence-to-Sequence Architecture with Application [6 Marks]

Encoder-Decoder is a neural network architecture used for sequence-to-sequence tasks (Seq2Seq), where input and output are both
sequences but may be of different lengths.

Architecture:

1. Encoder:
o Processes the input sequence (e.g., a sentence in English).
o Produces a fixed-size context vector (summary of input).
2. Decoder:
o Takes the context vector.
o Generates output sequence (e.g., a sentence in French), one word at a time.
3. Often uses LSTM or GRU units.

Application:

 Machine Translation: Translating "Hello" in English to "Bonjour" in French.

 Chatbots
 Summarization
 Speech-to-Text

18) Differentiate between Recurrent and Recursive Neural Network [5 Marks]

Feature Recurrent Neural Network (RNN) Recursive Neural Network

Structure Linear (sequential/time-based) Tree-like (hierarchical)
Input Type Sequential data (e.g., text, time series) Structured data (e.g., parse trees, expression trees)
Flow of Information One time step at a time Combines sub-parts recursively into parent nodes
Usage Language modeling, speech, translation Sentence parsing, scene understanding
Example Processing a sentence word-by-word Understanding grammar using syntax tree of a sentence