Artificial Intelligence

Convolution Neural Networks

Presented by
Dr.R.Anuradha
Associate Professor
Department of Computer Science and
Engineering
Sri Ramakrishna Engineering College
 Introduction to AI &ML &DL

06/01/2021 Artificial Intelligence for Data Analytics 2

 Traditional Programming
Data
Computer Output
Program

Machine Learning
Data
Computer Program
Output
06/01/2021 Artificial Intelligence for Data Analytics 3
06/01/2021 Artificial Intelligence for Data Analytics 4
 Supervised Learning

• Labeled data
• Predict Outcome/Future
• It infers a function from
labeled training data
consisting of a set of training
examples
• A supervised learning
algorithm takes a known set of
input data and known
responses to the data (output)
and trains a model to generate
reasonable predictions for the
response to new data

06/01/2021 Artificial Intelligence for Data Analytics 5

 Unsupervised Learning

• No labels
• Find hidden structure in data
• Used to draw inferences from
datasets consisting of input data
without labeled responses

06/01/2021 Artificial Intelligence for Data Analytics 6

 Reinforcement Learning

Problems involving an agent

interacting with an environment,
which provides numeric reward
signals

Goal: Learn how to take actions

in order to maximize reward

Artificial Intelligence for Data Analytics 7 06/01/2021

7
 Unsupervised “Weakly” supervised Fully supervised

Definition depends on task

06/01/2021 Artificial Intelligence for Data Analytics 8
 Generalization

Training set (labels known) Test set (labels unknown)

06/01/2021 Artificial Intelligence for Data Analytics 9

 Generalization
• Components of generalization error
– Bias: how much the average model over all training sets differ from the
true model?
• Error due to inaccurate assumptions/simplifications made by the
model
– Variance: how much models estimated from different training sets
differ from each other
• Underfitting: model is too “simple” to represent all the relevant
class characteristics
– High bias and low variance
– High training error and high test error
• Overfitting: model is too “complex” and fits irrelevant
characteristics (noise) in the data
– Low bias and high variance
– Low training error and high test error

06/01/2021 Artificial Intelligence for Data Analytics 10

 BIAS-VARIANCE TRADEOFF

06/01/2021 Artificial Intelligence for Data Analytics 11

06/01/2021 Artificial Intelligence for Data Analytics 12
 Supervised-Classification

• A classification problem is when the

output variable is a category, such as
“red” or “blue” or “disease” and “no
disease”
• A classification model attempts to draw
some conclusion from observed values
• Given one or more inputs a
classification model will try to predict
the value of one or more outcomes

06/01/2021 Artificial Intelligence for Data Analytics 13

 Regression
• It is a measure of the relation between
the mean value of one variable (e.g.
output) and corresponding values of
other variables (e.g. time and cost)
• A regression problem is when the
output variable is a real or continuous
value, such as “salary” or “weight”
• Many different models can be used, the
simplest is the linear regression
• It tries to fit data with the best hyper-
plane which goes through the points

06/01/2021 Artificial Intelligence for Data Analytics 14

 Examples

06/01/2021 Artificial Intelligence for Data Analytics 15

 Unsupervised -Clustering

• It is basically a type of unsupervised

learning method
• Clustering is the task of dividing the
population or data points into a number
of groups such that data points in the
same groups are more similar to other
data points in the same group and
dissimilar to the data points in other
groups
• It is basically a collection of objects on
the basis of similarity and dissimilarity
between them

06/01/2021 Artificial Intelligence for Data Analytics 16

 Clustering - Applications

06/01/2021 Artificial Intelligence for Data Analytics 17

 Deep Learning

• Deep learning is a sub-field of

machine learning
• Deep learning is based on the
concept of Artificial Neural
Networks(ANN), or computational
systems that mimic the way the
human bran functions
• Elements of DL
 Data, Task, Model, Loss
Function, Learning algorithm,
Evaluation

06/01/2021 Artificial Intelligence for Data Analytics 18

06/01/2021 Artificial Intelligence for Data Analytics 19
06/01/2021 Artificial Intelligence for Data Analytics 20
 Biological Neural Network Vs ANN

Biological Neural Network Artificial Neural Network

06/01/2021 Artificial Intelligence for Data Analytics 21

06/01/2021 Artificial Intelligence for Data Analytics 22
 Processing of ANN

06/01/2021 Artificial Intelligence for Data Analytics 23

 Activation Functions

Rectified Linear Unit (ReLu) Binary Step Logistic or Sigmoid

06/01/2021 Artificial Intelligence for Data Analytics 24

06/01/2021 Artificial Intelligence for Data Analytics 25
06/01/2021 Artificial Intelligence for Data Analytics 26
06/01/2021 Artificial Intelligence for Data Analytics 27
 Object Detection

06/01/2021 Artificial Intelligence for Data Analytics 28

Introduction to Convolution Neural
Networks

29
 Convolutional Neural Nets Background

● Gained in popularity after the “Alexnet” paper in 2012

● Type of feedforward neural network
● Used primarily for image recognition tasks
● Referred to as CNNs or ConvNets

30
31
Applications of CNNs

32
Applications of CNNs

33
Structure of a CNN

34
 CNN Structure
● Traditional neural networks are made up of an input layer, a series of hidden layers,
and an output layer
● Convolutional neural networks have convolutional layers, pooling layers, and fully
connected layers

35
What we see What Computers see

32 x 32 x 3

36
37
Convolutional Layer

38
 A convolutional layer
A CNN is a neural network with some convolutional layers
(and some other layers). A convolutional layer has a number
of filters that does convolutional operation.

Curve detector

A filter

06/01/2021 Artificial Intelligence for Data Analytics 39

Convolutional Layer

40
 Convolutional Layer
● Filters, from a high level, can be thought of as feature identifiers.

41
 Convolutional Layer
● Convolutional layers compute element wise multiplications between the values of
the input and the values of the filter.
Sample Image (5x5x1) Sample Filter (3x3x1)

1 1 1 0 0 1 0 1

0 1 1 1 0 0 1 0
0 0 1 1 1 1 0 1

0 0 1 1 0
0 1 1 0 0

42
Convolutional Layer

43
Convolutional Layer

44
 Convolutional Layer
● Multiply values in filter with the original pixel values of the image

45
 Convolutional Layer
● Let’s see the results after moving the filter

46
47
48
 Hyperparameters
◉ Depth – Number of filters
◉ Strides – Takes values 1, 2,3……
◉ Zero Padding – Reduce overfitting

49
 Strides OUTPUT SIZE:
N
(N-F)/stride+1
F
N
F
N=7, F=3
Stride=1 O/P=?
Stride=2 O/P=?
Stride=3 O/P=?

50
 Padding
◉ Common to Zero Pad the border
◉ Ex: 7 x 7 Input , 3 x 3 Filter, stride=1
◉ Pad with one pixel border, O/P=?

0 0 0 0 0 0 0 0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0 0 0 0 0 0 0 0

51
 Padding
◉ OUTPUT WILL BE THE SAME SIZE AS INPUT
◉ Common to see Convolution Layers with stride 1.
◉ Filters of size F x F and zero padding with (F-1)/2
◉ Preserve the size spatially
◉ Ex: F=3 => Zero pad with 1
F=5 => Zero pad with 2
F=7 => Zero pad with 3
Limits too fast shrinking of Image volume.

52
 Convolutional Layer
Calculating the output size for any given conv layer

O = Output height/length
W = Input height/length
P = Padding
S = Stride
K = Filer size

53
 QUESTION

◉ Input Volume (W) : 32 x 32 x 3

◉ 10 5 x 5 Filters with Stride 1, Pad 2
◉ Output Volume size: ?

54
 QUESTION

◉ Input Volume (W) : 10 x 10 x 3

◉ 3 x 3 Filters with Stride 2, No Padding
◉ Output Volume size: ?

55
 QUESTION

◉ Input Volume (W) : 32 x 32 x 3

◉ 10 5 x 5 Filters with Stride 1, 2 Padding
◉ No of parameters of this layer: ?

56
 Summary of Convolution layer

◉Accepts a volume of size W1 x H1 x D1

◉Requires four hyperparameters:
Number of filters K ,
their spatial extent F ,
the stride S ,
the amount of zero padding P .
Produces a volume of size W2 x H2 x D2
57
 Convolutional Layer
● In a CNN, there are normally multiple convolutional layers
● Filters in the first layer are designed to detect low level features such as curves and
edges
● Since the output of the first conv layer is used as input to the 2nd conv layer, the
input to the 2nd layer describes locations where low level features appear
● When we apply set of filters on top of that volume, the output activations will
represent higher level features

58
 Convolutional Layer
● Examples of higher level features
○ Semicircles (combination of a curve and a straight edge)
○ Squares (combination of several straight edges)
● As we go through more and more conv layers, outputs will represent more
and more complex features

59
60
61
 Pooling Layer
◉Pooling layer lies in-between successive Convolution layers in a
Conv.Net architecture.
◉Progressively reduce the spatial size of the representation to
reduce the amount of parameters and computation in the
network and control overfitting.

62
 Nonlinear Layer
● After each conv layer, we apply a nonlinear layer (or activation layer)
● Purpose is to introduce nonlinearity to a system that has just been computing
linear operations during the conv layers
● ReLU is the most popular activation function, followed by sigmoid and tanh
● ReLU stands for rectified linear unit, and is a type of activation function.
Mathematically, it is defined as y = max(0, x)

63
Non-linear Activation Function

64
 Dropout Layer
● This layer “drops out” a random set of activations by setting them to zero
● Purpose is to force the network to be robust and to alleviate problem of overfitting
● The network should be able to provide the right classification for a specific example
even when some activations are dropped
● Makes sure network isn’t getting too “fitted” to the training data

65
 Fully Connected Layer
● Final layer[s] in the network
● Exactly the same as the hidden layers we saw last week
● Takes in output volume of preceding layer and outputs N dimensional vector where
N is the number of classes.

66
Putting it all Together

67
CNN ARCHITECTURE

68
69
70
71
72
73
74
75
76
THANKS FOR YOUR
PATIENCE LISTENING

77