Machine Learning

(6CS4-02)

Unit-3 Notes

Vision of the Institute

To become a renowned centre of outcome based learning and work towards
academic, professional, cultural and social enrichment of the lives of individuals
and communities.

Mission of the Institute

M1- Focus on evaluation of learning outcomes and motivate students to
inculcate research aptitude by project based learning.
M2- Identify, based on informed perception of Indian, regional and global
needs, the areas of focus and provide platform to gain knowledge and solutions.
M3- Offer opportunities for interaction between academia and industry.
M4- Develop human potential to its fullest extent so that intellectually capable
and imaginatively gifted leaders can emerge in a range of professions.

Vision of the Department

To become renowned Centre of excellence in computer science and
engineering and make competent engineers & professionals with high ethical
values prepared for lifelong learning.

Mission of the Department

M1-To impart outcome based education for emerging technologies in the field
of computer science and engineering.
M2-To provide opportunities for interaction between academia and industry.
M3- To provide platform for lifelong learning by accepting the change in
technologies
M4- To develop aptitude of fulfilling social responsibilities.
Program Outcomes (PO)

1. Engineering knowledge: Apply the knowledge of mathematics, science, engineering

fundamentals, and an engineering specialization to the solution of complex
engineering problems.
2. Problem analysis: Identify, formulate, research literature, and analyze complex
engineering problems reaching substantiated conclusions using first principles of
mathematics, natural sciences, and engineering sciences.
3. Design/development of solutions: Design solutions for complex engineering problems
and design system components or processes that meet the specified needs with
appropriate consideration for the public health and safety, and the cultural, societal,
and environmental considerations.
4. Conduct investigations of complex problems: Use research-based knowledge and
research methods including design of experiments, analysis and interpretation of
data, and synthesis of the information to provide valid conclusions.
5. Modern tool usage: Create, select, and apply appropriate techniques, resources, and
modern engineering and IT tools including prediction and modeling to complex
engineering activities with an understanding of the limitations.
6. The engineer and society: Apply reasoning informed by the contextual knowledge to
assess societal, health, safety, legal and cultural issues and the consequent
responsibilities relevant to the professional engineering practice.
7. Environment and sustainability: Understand the impact of the professional
engineering solutions in societal and environmental contexts, and demonstrate the
knowledge of, and need for sustainable development.
8. Ethics: Apply ethical principles and commit to professional ethics and responsibilities
and norms of the engineering practice.
9. Individual and team work: Function effectively as an individual, and as a member or
leader in diverse teams, and in multidisciplinary settings.
10. Communication: Communicate effectively on complex engineering activities with the
engineering community and with society at large, such as, being able to comprehend
and write effective reports and design documentation, make effective presentations,
and give and receive clear instructions.
11. Project management and finance: Demonstrate knowledge and understanding of the
engineering and management principles and apply these to one’s own work, as a
member and leader in a team, to manage projects and in multidisciplinary
environments.
12. Life-long learning: Recognize the need for, and have the preparation and ability to
engage in independent and life-long learning in the broadest context of technological
change.
Program Educational Objectives (PEO)
1. To provide students with the fundamentals of Engineering Sciences with more
emphasis in Computer Science &Engineering by way of analyzing and exploiting
engineering challenges.
2. To train students with good scientific and engineering knowledge so as to
comprehend, analyze, design, and create novel products and solutions for the real
life problems.
3. To inculcate professional and ethical attitude, effective communication skills,
teamwork skills, multidisciplinary approach, entrepreneurial thinking and an
ability to relate engineering issues with social issues.
4. To provide students with an academic environment aware of excellence,
leadership, written ethical codes and guidelines, and the self-motivated life-long
learning needed for a successful professional career.
5. To prepare students to excel in Industry and Higher education by Educating
Students along with High moral values and Knowledge

Program Specific Outcomes (PSO)

PSO1: Ability to interpret and analyze network specific and cyber security issues, automation
in real word environment.
PSO2: Ability to Design and Develop Mobile and Web-based applications under realistic
constraints.
Course Outcome:

CO1: Understand the concept of machine learning and apply supervised

learning techniques.
CO2: Illustrate various unsupervised leaning algorithm for clustering, and
market basket analysis.
CO3: Analyze statistical learning theory for dimension reduction and model
evaluation in machine learning.
CO4: Apply the concept of semi supervised learning, reinforcement learning and
recommendation system.

CO-PO Mapping:

CO PO1 PO2 PO3 PO4 PO5 PO6 PO7 PO8 PO9 PO10 PO11 PO12
Understand the concept of
machine learning and apply
supervised learning techniques.
3 3 3 3 2 1 1 1 1 2 1 3

Illustrate various unsupervised

leaning algorithm for clustering, 3 3 3 2 2 1 1 1 1 1 1 3
and market basket analysis.
Analyze statistical learning
theory for dimension reduction
and model evaluation in 3 3 3 3 2 2 2 2 1 2 2 3
machine learning.
Apply the concept of semi
supervised learning,
reinforcement learning and
recommendation system. 3 3 3 3 2 1 1 1 1 2 1 3
SYLLABUS:
LECTURE PLAN:

Unit No./
Total Lect. Lect.
Topics
Lecture Reqd. No.
Reqd.
1. Introduction to subject and scope 1 1
2. Introduction to learning, Types of learning and Applications 1 2
3. Supervised Learning 1 3
4. Linear Regression Model 1 4
Unit-I 5. Naïve Bayes Classifier 1 5
(10) 6. Decision Tree 1 6
7. K-nearest Neighbor 1 7
8. Logistic Regression 1 8
9. Support Vector Machine 1 9
10. Random Forest Algorithm 1 10

BC-1 Gradient Descent 1 11

1. Introduction to clustering, K-mean clustering 2 12

2. Hierarchical Clustering 1 14
3. Probabilistic Clustering 1 15
Unit-II
4. Association Rule Mining 1 16
(8)
5. Apriori Algorithm 1 17
6. f-p Growth Algorithm 1 18
7. Gaussian Mixture Model 1 19
1. Feature Extraction- PCA and SVD 3 22
2. Feature Selection- Feature Ranking and Subset Selection 2 24
Unit-III
3. Filter, Wrapper and Embedded Methods 1 25
(8)
4. Evaluating Machine Learning Algorithms 1 26
5. Evaluating Model Selection 1 27
1. Semi supervised learning: Markov Decision Process (MDP) 2 29
2. Bellman Equations 1 30
3. Policy Evaluation using Monte Carlo 1 31
Unit- IV 4. Policy iteration and Value iteration 1 32
(8) 5. Q-Learning 1 33
6. State-Action-Reward-State-Action (SARSA) 1 34

7. Model-based Reinforcement Learning 1 35

 1. Recommendation system: Collborative Filtering 1 36
2. Content based filtering 1 37
3. Artificial neural network 1 38
Unit- V
4. Perceptron 1 39
(8)
5. Multilayer network 1 40
6. Backpropagation 1 41
7. Introduction to Deep learning. 2 42

BC-2 Genetic Algorithms 1 44

Text Book:
Machine learning- Tom M Mitchell
aati Aeluchion
eatu akeetion ishe Proc
ofeducmothe imput anuale t
tb h a Madel
data .

b umq m t releam
Kilhion t
The 9eal of Peawu
Nachune earin is to ind d bestsa F
ealw tha dlous ons t o build Madolg e
&sach on e Can
&uch m technques OpHming oa Hodes
&anau
h Mahos )Prven-la
(hrappen ethod
Umproel
mLadded MaHad oceunay
3)Raduca dro

fenemnoma

te
Machima

S o fal Se Aqaritm

eatns
e
rmne
eiiq
qt-e
tedh
nwgn
pt
naualle
tuse3 e ond
This
Mehed
v
auia
abbl
le
sdug
&else h e of
rolen to
ofeatuy is indepenoan
&eloc'o m
clarsiien used uLejul
How Huch
Mcns
ankn9 x psetco
eas oataa
amd fnpontan
beor cla non
ban
bancally
waiablas as a Fve frocen4
re froce
f
SubSo s of Hhe holem redicheY
redic
ndebendlevdy
Tn +his Nehoel teoekunas ane droPhee balad
Om heY Teloton +H +ha a o How a
a CYelasinto he outut
K a m R

hme o ofGndnd
timsrad Colc

Jm Bosk cospien ule dophed he

he Calo

Cn a siml eladnh'on.
Colmm baad

Lwrhe Mehad tn t a
we &pli olata
SebJes Cund hcun adel na hid Based
Cm he Outa of the Madol ú L add a
ack
no

Asroet eates hecda the Madal aap

Sa of Peatas

emaal

Subde

Alaerm

fenfeymane o
t e r eonyple

inYaphen Helhod
mochlne Cind odrut
train he
dpenem eaty t o
he&auLl tccodun to Owpuut
Nama Noo f Condi Wm Celor
of oclk

Noma Cmd NO of Hmes

read

neod omol Comdi

Codi hens
hens

NOmm, No of imey
efLen w s Chocle

maadds Hahad Thus Mhed Combines

he qual'Hes of boh l t Cmd Wrappen

M e h a d t o Cnaal
ma hos+ Aubl-

2 f aalue

Gerurde
Subdet

AhgonmF
fentemanee

Mmdel rain gmd ceck hAelocl Ho

The dienem-Saubsels
eno
of
accuncc
bes Cmmom homn
 teatuna Aclechomn Mehioud

Sapni4ep\ umbubenUNAod

wYabhu embuddoo
Mood Mahod Matacl

Rew henchic Decihem

Coekioiot ANOVA elinmakyn
Algnihn
hee

t Hethad
(1 C Squane Corelohn Coekc C
easm
CoeHiien-
0)
11) ANoV +
Whrapben Melhod Peatus eliminahion

(1) Recwtiue

Gyemahic Mgomm
*einbedded Heaol

Raqula ngeu-on
( ) LosAo
((1) Dociicn TTe
w
heauu'a
ab le
bl es
s

owau
one
On mhut- A a l hbn Macu!.
saluhbn
Macde!.
Basko
ea
a Can Choase
Nmenica! impul umenioal crpl S
Nuneai ri cal imbl CalcqmCal Oulfa
Cotrgoncal inpud2 me u co otpd

Calegemcal tnpeul 2 Catrgmca ouh

inpul
aniab

Numen cal CCati gen'cal

OuHocs- Uarucl

CoTceema Mume uccd Caugemta

an sens ANOV ANOVA C

Sqared

Spapan's Kenlals keadal's Mtuel

memohin

Chi Squay Tes

1es weo to
chiaalt anp1-cs setalecl
tuu naniables

och omen
relahndhip»
 Table of obse d aluue
ualulioMuiddle Hig Bochalor's HaulePhD Tatal
Houtan
tatus clasclas
Neur Ho 18 36 2 6

Maued 2 36 36 2)
DWorceo 9 30

Wielowed 3 9 6 3 30
6 33 3
Tatal 9

value
Toble os t<picko
outitHddl Badaor's Masleu
fhDT
claa cla Cy
Statiy

Jeuwl
90x39 oxgo 2 2 262 99
Mantd
2

Maanued 29S S y2 27 16s

Divarwd 3 9 6 4 Sy33

Lwicosed 39 8-
Teodo
 chi Seare (n) = obseac ualu-2.pocled val

Eoced ualue
Obsened
nauue pacbed(0-)(o-c (o-e
aluLe
.11+
3 39-69
3 39
24 B1
21 2S 2 - 2
7
6 2 -1.2 SI8y 3-2
9-9 39
IC2 S3
12 19 S -7 S
S6-2S 2-88
36 S 9 18

36
uS
2
2 6S S 2o 25 1-22
3 21 , 4) 1-13
O
O6 0-36 .0
S.9 -2-y S16 06
3:3 -03 0-09 O 027
3-9 0-9 6-20

9 8.4 06 0 36
-2-y S.16 06
3 o3 2

2
Catcaltecd23 s+
2 3.s
 Degrec f edom=((olcn (8
( S - ) (u)
12
a d a alue of 0-oS
c h e kwd(&) =
tublan 21a3

= 23 sT
Caldatrd

22
Caluloo tabular
Gumd o c t a l
N l hu 4

altvnatehykoha

Cnd
asilal tt
Rlahi ond beueem

(ua 'cahon.
 The n u Jaal Comoma UO

C
G1 +O 19 2

192+ O 1
Singula Valua DecomosiHm
SvD i s a0idely
wed tocdruena to
docommaosa amahn x nto
seenal omponom mamAs, xpo slma momy
Of the
ituiq Prope s of
Onral mah x
Th punp 0se. of
doconpo siHon is to
inadan volu
-seducs olota
a laae Mmben
a

olata Sol ontaim q

Contaimna
soCntly
CovTaln a l o s g
alueL bul didh
chems th uasu abi
oyen m the i v a olata

Am
wner
Ond aeksngelon. clu'oaenad mamx of Swngulan aluey.
A mm mxn mXn .
U.u=I
V.v =T OhogUnod
Topey
 A = u ev
tA vS' U
A A vEy)yEVT

ATA V£'gV
(us (ye'o) bocauus i. V-
No R A =

AA = UsËU |
3 11

A- 34J2X3 A. A-V
A A=U

3X2

A A a
-13

xI 3* +Ix3 +1x1
3x3 + 1X)+
IX -) + 33 +1X)
x3 + 3X1+1 XX

2X2

A-AE =0
CN-) 1

L (t-
(11-a) = o
J-+) -d-) = o
 (12-) 0 O-do

A12 d2-l0

d-10
-d) 1-10)
1-12)
1+12F
J41)2
Onoemalzaicn '
Gom schmidt rhopmaliggk on ro CO

AA 1 x
2 a
2 2
J3x3
-AT)1 = 0
o-) o

o-a)

3 s hext C Mahoal

S S3
2
SA S d-S3
umo Clelamlnan ofFA ).
trace (A O dicoeds
 tac E Sun of cliaoe nad vodu
o + o +2

dutovniaunl) 1o/
lo20 o+2/o

l o [ ] +22

= O
D uO - u o

unos of diayemous
OO 2

ter diaspralvulu ((o) (lo) c2)

3 - 22d+12o d = o

AC-22d +120) = O d o
-22d +120 = O

A - 2 d -Od+120 O

ACd2)-1o(2) =o
da=12
da10
 12
d12 to-d) o 2
lo-a)4 V

(2-
-2 O 2
2
2 u

(A-aT)x =d
hau &o

-2 2
-2

2 4-10

rames u

2
T
mwauy
A
When d2-10
22 4

2 8 V
-
ntole
M

N
O