Scribd
Upload
46 views13 pages

Pattern Recognition Techniques

A small presentation on Pattern Recognition techniques for Electrical and Communication Engineering.

Uploaded by

Sonu Gangwani
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
46 views13 pages

Pattern Recognition Techniques

A small presentation on Pattern Recognition techniques for Electrical and Communication Engineering.

Uploaded by

Sonu Gangwani
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 13

PATTERN

RECOGNITION
TECHNIQUES

PRINCIPAL COMPONENT
ANALYSIS

NEXT
Introduction to PCA & its Purpose
 Large datasets are increasingly widespread in many
disciplines. In order to interpret such datasets, methods are
required to drastically reduce their dimensionality in an
interpretable way, such that most of the information in the
data is preserved.
 PCA is one of the oldest and most widely used for this
purpose.
 Its idea is simple to reduce the dimensionality of a dataset,
while preserving as much ‘variability’ as possible.
 It is adaptive in another sense too, since variants of the
technique have been developed that are tailored to various
different data types and structures.
Geometric intuition
 Let’s take an example of a dataset with features f1
and f2.The data is standardized i.e. mean centered
and variance is 1.
 We plot the points in the feature space.
 Then we rotate our axes to find the best principal
component(z1 and z_2) such that the variance of
x_i projected onto z1 is maximal.
 We drop z_2 later. So z_2 becomes our only
needed direction.
 In PCA we are not changing the points present in
space. We change the axes , then coordinates of
each point will change but the actual location of
the points remains unchanged.
Mathematical objective function

 Here the points are plotted in the f1 and f2 feature space,


we rotate the axes and find the unit vector u1. we project
the points on this vector and calculate the variance
observed.
Mathematical objective function
Mathematical objective function
PCA: Distance Minimization

 Here d1 and d2 are the distances of the points x1 and x2 from the unit vector.
 In the distance minimization interpretation of the PCA we have to minimize the distance of the points from the
unit vector selected so as to increase the spread on that particular axis only.
PCA: Dimensionality reduction

 Dimensionality reduction is the process of reducing the


number of random variables under consideration, via
obtaining a set of principal variables .It can be divided
into feature extraction and feature selection.
 Feature Selection: In this method we select the best
features from a set of features provided in the data set.
We do not create new features, instead we choose the
most important features from the given set of features
and hence we remain with lesser dimensions or lesser
features to work with.
 Feature Extraction: In this method we create new
features, and these features are not present in our
original feature set. These features are not interpretable.
PCA: Dimensionality reduction
Following are the essential points which explain why we
need Dimensionality reduction:
1. It reduces the time and storage space required.
2. Removal of multi-collinearity improves the
interpretation of the parameters of the machine
learning model.
3. It becomes easier to visualize the data when reduced to
very low dimensions such as 2D or 3D. This is because
in general we need to deal with very high dimensional
data i.e. data with 100s and 1000s of dimensions so in
that case it wont be possible to visualize that data and
to work upon such data we may not have enough
computation power as well.
4. It avoids the curse of dimensionality.
PCA: Limitations

 PCA cannot be used for sparse data. For sparse data


we use SVD .
 If features are completely uncorrelated, the lower
dimensional representation obtained using PCA
would not preserve much of the variance in the
original data and hence would be useless. PCA
works fine if a subset of features are correlated.
Before we apply PCA we try to remove outliers.
 PCA assumes that the principal components are
orthogonal.
 Since PCA is a Feature extraction technique so the
new features which are formed do not make any
sense i.e. are not interpretable.
 PCA doesn’t work well on the datasets with the plots
as shown below:
PATTERN
RECOGNITION
TECHNIQUES

THANK YOU

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy