Lecture 1

Lecture 1: Introduction to Deep Learning
Md. Shahriar Hussain

ECE Department, NSU
North South University CSE465 Md. Shahriar Hussain

Machine Learning and AI
North South University CSE465 Md. Shahriar Hussain 2

Artificial Intelligence
• Artificial Intelligence
– The science of making machine that think like people and act like people
– ability of a computer or machine to mimic or imitate human intelligent behavior and
perform human-like tasks
• Artificial Intelligence aims to provide approximate

solutions to computationally complex problems such as
vision, speech, natural language processing (NLP)
• Machine Learning is a sub set of AI

Machine Learning
• Machine Learning is the science (and art) of programming computers so they

can learn from data.
• More general definition:

Machine Learning is the field of study that gives computers the ability to learn
without being explicitly programmed.
—Arthur Samuel, 1959

Machine Learning
• Machine Learning is new paradigm of AI
• Good Old-fashioned AI : human domain experts write set of rules for

computer programs
• In Machine Learning computer programs can automatically learn the rules

from data. (Inductive Process)

Traditional Programming
• Traditional Programming: writing a set of RULES to find ANSWERS from

DATA

The ML Approach
• Machine Learning: Use DATA and ANSWERS to learn the underlying set of
RULES

Types of Machine Learning Systems
 Supervised Learning
 Unsupervised Learning
 Semisupervised Learning

Supervised Learning
• ML algorithm uses the training dataset to create model/assumption/hypothesis

which can be used to predict (unseen/new data)
• Training data fed to algorithm includes the desired answers/solutions (labels)
• ML algorithm iteratively makes predictions on the training data and is corrected by
the output/supervision of teachers

Supervised Learning

Supervised Learning

Unsupervised Learning
• Unlabeled dataset. System learns without direct human supervision
• Unsupervised learning algorithms take a set of data that contains only

inputs, and find structure/patterns in the data, like grouping or clustering of
data points.
• The algorithms therefore learn from test data that has not been labeled,
classified or categorized
• Instead of responding to feedback, unsupervised learning algorithms identify

commonalities in the data and react based on the presence or absence of
such commonalities in each new piece of data.
• Useful for business intelligence

Unsupervised Learning

Semi-Supervised learning
• In Semi-Supervised learning, the training data contains both labeled and

unlabeled data
– We have a large dataset. Manual labeling of the entire dataset is laborious and expensive
– One solution could be to label a sample of the dataset and train the labeled portion to
create a model. This would however mean that we are not fully utilizing the larger dataset
we have and thus the model that we create may be less robust
– A potential solution: (1) Label a sample of the large dataset, (2) train a model using this
labeled portion, (3) use the model to predict the unlabeled portion (pseudo-labeling), (4)
train using the entire dataset
– e.g., Google Photos will cluster similar faces, and ask the user if they are the same
person

Machine Learning: Inductive Process
• How does the induction based technique work in ML?
• Consider the blue data points that are generated by an unknown function (green
line).
• To learn this function from data by ML, at first we need to make an inductive
hypothesis about the function (red line)
Inductive hypothesis: Inductive hypothesis:

the function is linear the function is nonlinear (polynomial)

Machine Learning: Inductive Process
• Without the inductive hypothesis, there is no learning
• The hypothesis is our model about the hidden pattern.
Linear Polynomial
Regression Model Regression Model

Machine Learning
• In ML, we manually define the representations
• We design a set of inductive hypotheses that comprise the hypothesis

space.
– Example, linear and a finite set of polynomials.
• The ML algorithm “finds” the optimal hypothesis

Deep Learning
• Subset of ML - loosely mimics

structure/function of human brain
• Unlike traditional ML, does not require

manual feature extraction
• Keeps getting better with more data

(typically)

Deep Learning
• In DL, we don’t have to manually design optimal representations of input

data.
• DL “learns” representations.
• What makes DL powerful is that it learns representations in hierarchical

fashion.
– Low-level representation to high-level representation

Deep Learning
• In this hierarchical model of DL, each computational layer learns

increasingly complex representation of the input.
• But how do we implement this hierarchical system?
• We turn to biological neural networks that uses hierarchical information
processing system

Artificial Neural Network
• An Artificial Neural Network (ANN) or Neural Network is a computational

model that is loosely based on the neural structure of the biological brain

Biological Neuron
• Biological neurons can be understood as the subunits of a neural network

in a biological brain.
• Here, the signals of variable magnitudes arrive at the dendrites.

Biological Neuron
• Those input signals are then accumulated in the cell body of the neuron.
• If the accumulated signal exceeds a certain threshold, a output signal is

generated, which will be passed on by the axon.

Biological Neuron
• Based on the function of a biological neuron, following three rules can be used to
design a computation model.
1. Output is binary (either 0/1, -1/+1)

2. Input signals produce a cumulative influence on the output
3. Each input signal has a weight

Artificial Neuron
• We may capture these 3 rules in a simple artificial neuron.

1. Output is binary (either 0/1, -1/+1)
2. Input signals produce a cumulative influence on the output
3. Each input signal has a weight

Artificial Neuron
• The neuron receives multiple input signals.

• If the sum of the input signals exceed a certain threshold it either returns a
signal or remains “silent”

Artificial Neuron
• For a given input signal, some of the connection weights need to be large
and some need to be small to produce the desired output (to fire or not).
• Thus, an algorithm was required to learn the values of the weights w

Perceptron
• The Perceptron was the first pragmatic attempt to mimic biological

neural network for machine based learning
• The Perceptron model was used to solve classification problems.
• It was observed shortly that Perceptron can do only trivial classification

(on linearly separable data set).

Neural Network
• In 1986 the Multi-layer Perceptron (MLP)

model was invented
• A MLP is a collection of artificial neurons,

organized in a layered architecture, which
gives the name Neural Networks (NNs).
• An NN is organized in three interconnected

layers:
– input,
– hidden that may include more than one layer,
– output.

Lecture 1

Uploaded by

Copyright:

Available Formats

Lecture 1

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Lecture 1

Uploaded by

Copyright:

Available Formats

Lecture 1: Introduction to Deep Learning

Md. Shahriar Hussain

North South University CSE465 Md. Shahriar Hussain

North South University CSE465 Md. Shahriar Hussain 2

• Artificial Intelligence aims to provide approximate

• Machine Learning is a sub set of AI

North South University CSE465 Md. Shahriar Hussain 3

• Machine Learning is the science (and art) of programming computers so they

• More general definition:

North South University CSE465 Md. Shahriar Hussain 4

• Machine Learning is new paradigm of AI

• Good Old-fashioned AI : human domain experts write set of rules for

• In Machine Learning computer programs can automatically learn the rules

North South University CSE465 Md. Shahriar Hussain 5

• Traditional Programming: writing a set of RULES to find ANSWERS from

North South University CSE465 Md. Shahriar Hussain 6

North South University CSE465 Md. Shahriar Hussain 7

North South University CSE465 Md. Shahriar Hussain 8

• ML algorithm uses the training dataset to create model/assumption/hypothesis

North South University CSE465 Md. Shahriar Hussain 9

North South University CSE465 Md. Shahriar Hussain 10

North South University CSE465 Md. Shahriar Hussain 11

• Unlabeled dataset. System learns without direct human supervision

• Unsupervised learning algorithms take a set of data that contains only

• Instead of responding to feedback, unsupervised learning algorithms identify

• Useful for business intelligence

North South University CSE465 Md. Shahriar Hussain 12

North South University CSE465 Md. Shahriar Hussain 13

• In Semi-Supervised learning, the training data contains both labeled and

North South University CSE465 Md. Shahriar Hussain 14

Inductive hypothesis: Inductive hypothesis:

North South University CSE465 Md. Shahriar Hussain 15

North South University CSE465 Md. Shahriar Hussain 16

• In ML, we manually define the representations

• We design a set of inductive hypotheses that comprise the hypothesis

– Example, linear and a finite set of polynomials.

• The ML algorithm “finds” the optimal hypothesis

North South University CSE465 Md. Shahriar Hussain 17

• Subset of ML - loosely mimics

• Unlike traditional ML, does not require

• Keeps getting better with more data

North South University CSE465 Md. Shahriar Hussain 18

• In DL, we don’t have to manually design optimal representations of input

• What makes DL powerful is that it learns representations in hierarchical

North South University CSE465 Md. Shahriar Hussain 19

• In this hierarchical model of DL, each computational layer learns

North South University CSE465 Md. Shahriar Hussain 20

• An Artificial Neural Network (ANN) or Neural Network is a computational

North South University CSE465 Md. Shahriar Hussain 21

• Biological neurons can be understood as the subunits of a neural network

North South University CSE465 Md. Shahriar Hussain 22

• If the accumulated signal exceeds a certain threshold, a output signal is

North South University CSE465 Md. Shahriar Hussain 23

1. Output is binary (either 0/1, -1/+1)

North South University CSE465 Md. Shahriar Hussain 24

• We may capture these 3 rules in a simple artificial neuron.

North South University CSE465 Md. Shahriar Hussain 25

• The neuron receives multiple input signals.

North South University CSE465 Md. Shahriar Hussain 26