Scribd
Upload
18 views3 pages

Deep Learning Notes

Deep Learning is a subset of machine learning that utilizes neural networks to solve complex tasks such as image recognition and speech translation. A neural network consists of layers of neurons, where each connection has a weight that is adjusted during training to minimize prediction error. Multi-Layer Perceptrons (MLPs) are used to learn complex patterns, employing techniques like backpropagation and gradient descent for effective learning.

Uploaded by

e.work3mail
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
18 views3 pages

Deep Learning Notes

Deep Learning is a subset of machine learning that utilizes neural networks to solve complex tasks such as image recognition and speech translation. A neural network consists of layers of neurons, where each connection has a weight that is adjusted during training to minimize prediction error. Multi-Layer Perceptrons (MLPs) are used to learn complex patterns, employing techniques like backpropagation and gradient descent for effective learning.

Uploaded by

e.work3mail
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 3

Notes : deep learning

Saturday, May 24, 2025 9:55 PM

What is Deep Learning?


Deep Learning is a subset of machine learning that uses algorithms inspired by the human brain —
called neural networks — to model and solve complex tasks like image recognition, speech
translation, and game playing.
Key Idea: Learn patterns in data through multiple layers.
Neural Network Basics
What is a Neural Network?
A neural network is made up of neurons (nodes) arranged in layers:
• Input Layer – takes features (like pixels or measurements)
• Hidden Layers – extract patterns
• Output Layer – produces prediction
Each connection has a weight, and each neuron has a bias. The network learns by adjusting these to
reduce error.
What is a Perceptron?
A perceptron is the simplest building block of a neural network. It takes multiple inputs, multiplies
each by a weight, adds a bias, and passes the result through an activation function to produce an
output.
Formula:
z=w1x1+w2x2+…+wnxn+b
y^=Activation(z)
Structure of a Perceptron
Inputs: x1 x2 x3
│ │ │
w1│+ w2│+ w3│
└─────┬──────┘
▼
Weighted Sum + Bias
▼
Activation Function (e.g. Step, ReLU)
▼
Output
Activation Functions
These add non-linearity to the network, enabling it to learn complex patterns.
Function Formula Output Range Notes
Step 1 if z > 0 else 0 0 or 1 Used in original perceptron
Sigmoid 1/1+e−z (0, 1) Good for probabilities
Tanh ez−e−z /ez+e−z (-1, 1) Zero-centered
ReLU max(0, z) [0, ∞) Most common in DL

Perceptron Learning Rule


We adjust weights to reduce prediction error:
wi := wi + η(y−y^)xi
b := b+η(y−y^)
Where:
• η= learning rate (small number like 0.01)
• y= actual output
• y^= predicted output
Limitation: XOR Problem
A single perceptron cannot solve XOR because XOR is not linearly separable.
Solution? ➤ Multi-Layer Perceptrons (MLPs)
What is an MLP?
A Multi-Layer Perceptron (MLP) is a neural network with:
• An input layer
• One or more hidden layers
• An output layer
Each layer is made of neurons (perceptrons), fully connected to the next layer.
Why Use Multiple Layers?
• One layer: Can only learn linear boundaries (e.g., line)
• Multiple layers: Can learn complex and non-linear functions (e.g., XOR, image patterns)
Activation Functions in Hidden Layers
Function Why Use It?
ReLU Fast & helps avoid vanishing gradient
Tanh Zero-centered output
Sigmoid Good for probabilities, but prone to vanishing gradients
Backpropagation + Gradient Descent
This is how the network learns.
Steps:
1. Compute the loss.
2. Backward pass: Use backpropagation to compute gradients of weights.
3. Update weights using gradient descent.
w:=w−η (∂Loss/∂w )
Forward Propagation
This is how input flows through the network to generate predictions.
Steps:
1. Multiply inputs by weights.
2. Add bias.
3. Apply activation.
4. Pass result to next layer.
Repeat until the output layer gives the final result y^.
Vocabulary:
• Backpropagation
It’s the brain of learning.
• After seeing how wrong the output is (using the cost function), backpropagation goes
backward through the network.
• It calculates how much each weight contributed to the error, so it can be adjusted properly.
• Gradient Descent
This is the method the model uses to learn.
• It adjusts the weights in small steps to reduce the error.
• Like walking downhill to reach the lowest point (minimum cost).
• Learning Rate
It controls how big the steps are in gradient descent.
• If too small, training is slow.
• If too big, the model might skip the best solution.
• Batches
We don't feed the entire dataset at once into the network. Instead, we split it into smaller parts
called batches.
• Helps with memory and speeds up training.
• Epochs
One epoch means the model has seen all the data once.
• Training usually takes many epochs so the model can learn well.

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy