Click on Subject/Paper under Semester to enter.

Professional English Discrete Mathematics Environmental Sciences

Professional English - - II - HS3252 - MA3354 and Sustainability -
I - HS3152 GE3451
Digital Principles and
Statistics and Probability and
Computer Organization
Matrices and Calculus Numerical Methods - Statistics - MA3391
- CS3351
- MA3151 MA3251
3rd Semester
1st Semester

4th Semester
2nd Semester

Database Design and Operating Systems -

Engineering Physics - Engineering Graphics
Management - AD3391 AL3452
PH3151 - GE3251

Physics for Design and Analysis of Machine Learning -

Engineering Chemistry Information Science Algorithms - AD3351 AL3451
- CY3151 - PH3256
Data Exploration and Fundamentals of Data
Basic Electrical and
Visualization - AD3301 Science and Analytics
Problem Solving and Electronics Engineering -
BE3251 - AD3491
Python Programming -
GE3151 Artificial Intelligence
Data Structures Computer Networks
- AL3391
Design - AD3251 - CS3591

Deep Learning -
AD3501

Embedded Systems
Data and Information Human Values and
and IoT - CS3691
5th Semester

Security - CW3551 Ethics - GE3791

6th Semester

7th Semester

8th Semester

Open Elective-1
Distributed Computing Open Elective 2
- CS3551 Project Work /
Elective-3
Open Elective 3 Intership
Big Data Analytics - Elective-4
CCS334 Open Elective 4
Elective-5
Elective 1 Management Elective
Elective-6
Elective 2
 All Computer Engg Subjects - [ B.E., M.E., ] (Click on Subjects to enter)
Programming in C Computer Networks Operating Systems
Programming and Data Programming and Data Problem Solving and Python
Structures I Structure II Programming
Database Management Systems Computer Architecture Analog and Digital
Communication
Design and Analysis of Microprocessors and Object Oriented Analysis
Algorithms Microcontrollers and Design
Software Engineering Discrete Mathematics Internet Programming
Theory of Computation Computer Graphics Distributed Systems
Mobile Computing Compiler Design Digital Signal Processing
Artificial Intelligence Software Testing Grid and Cloud Computing
Data Ware Housing and Data Cryptography and Resource Management
Mining Network Security Techniques
Service Oriented Architecture Embedded and Real Time Multi - Core Architectures
Systems and Programming
Probability and Queueing Theory Physics for Information Transforms and Partial
Science Differential Equations
Technical English Engineering Physics Engineering Chemistry
Engineering Graphics Total Quality Professional Ethics in
Management Engineering
Basic Electrical and Electronics Problem Solving and Environmental Science and
and Measurement Engineering Python Programming Engineering
 lOMoARcPSD|45333583

www.BrainKart.com

Vel Tech Rangarajan Dr.Sagunthala R&D Institute of

Science and Technology

School of Computing

Department Of Computer Science andEngineering

1152CS234 – Machine Learning

Lab Manual

Year :  WINTER

FACULTY NAME :

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 1 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

CONTENTS

Sl.No. Experiments Page

No.
Apply and demonstrate the FIND-S algorithm for finding the most specific 7
1
Hypothesis based on a given set of training data samples.

Implement and demonstrate the Candidate-Elimination algorithm to output a 10

2
description of the set of all hypotheses consistent with the training examples.

Write a program to demonstrate the working of the decision tree based on ID3 12
3
algorithm. Use an appropriate data set for building the decision tree and apply this
knowledge to classify a new sample.
Write a program to implement the Gaussian mixture model. Compute the accuracy
4 16
of the classifier, considering few test data sets.
Apply back propagation neural network on image data. The idea is to build a 19
5
Artificial Neural Network model that can effectively analyze and extract features
from an image.
Assuming a set of documents that need to be classified, use the Radial basis 23
6
function Classifier model to perform this task. Calculate the classification rate,
accuracy,
precision, and recall for your data set.
Write a program to construct a Hidden markov model for a medical data. Use this 30
7
model to demonstrate the diagnosis of medial data.

Apply EM algorithm to cluster a set of data. Use the same data set for clustering
37
8 using K- Means algorithm. Compare the results of these two algorithms and
comment on the quality of clustering. You can add Java/Python ML library
classes/API in the program.
Apply LDA algorithm to select the appropriate data from the given data set. Use 40
9
XG boost algorithm for classification.

Build a classification model that can effectively analyze and extract features from
10 46
an image. Apply PCA algorithm to find the appropriate feature.
Use case Implementation
11

2
https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 2 / 57
 lOMoARcPSD|45333583

www.BrainKart.com
1152CS234 ML LAB

Introduction

Machine learning

Machine learning is a subset of artificial intelligence in the field of computer science that often
uses statistical techniques to give computers the ability to "learn" (i.e., progressively improve
performance on a specific task) with data, without being explicitly programmed. In the past
decade, machine learning has given us self-driving cars, practical speech recognition, effective
web search, and a vastly improved understanding of the human genome.

Machine learning tasks

Machine learning tasks are typically classified into two broad categories, depending on whether
there is a learning "signal" or "feedback" available to a learning system:

1. Supervised learning: The computer is presented with example inputs and their
desiredoutputs, given by a "teacher", and the goal is to learn a general rule that maps inputs to
outputs. As special cases, the input signal can be only partially available, or restricted to special
feedback:

2. Semi-supervised learning: the computer is given only an incomplete training signal:

atraining set with some (often many) of the target outputs missing.

3. Active learning: the computer can only obtain training labels for a limited set of instances
(based on a budget), and also has to optimize its choice of objects to acquire labels for. When
used interactively, these can be presented to the user for labeling.

4. Reinforcement learning: training data (in form of rewards and punishments) is given only as
feedback to the program's actions in a dynamic environment, such as driving a vehicle or playing
a game against an opponent.

Dept. of CSE, Page 2

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 3 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

5. Unsupervised learning: No labels are given to the learning algorithm, leaving it on its own to
find structure in its input. Unsupervised learning can be a goal in itself (discovering hidden
patterns in data) or a means towards an end (feature learning).

Machine Learning Applications

In classification, inputs are divided into two or more classes, and the learner must produce a
model that assigns unseen inputs to one or more (multi-label classification) of these classes. This
is typically tackled in a supervised manner. Spam filtering is an example of classification, where
the inputs are email (or other) messages and the classes are "spam" and "not spam".

In regression, also a supervised problem, the outputs are continuous rather than discrete.
In clustering, a set of inputs is to be divided into groups. Unlike in classification, the groups are
not known beforehand, making this typically an unsupervised task.

Density estimation finds the distribution of inputs in some space.

Dimensionality reduction simplifies inputs by mapping them into a lower dimensional space.
Topic modeling is a related problem, where a program is given a list of human language
documents and is tasked with finding out which documents cover similar topics.

Machine learning Approaches

1. Decision tree learning

Decision tree learning uses a decision tree as a predictive model, which maps observations about
an item to conclusions about the item's target value.

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 4 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

2. Association rule learning

Association rule learning is a method for discovering interesting relations between variables in
large databases.

3. Artificial neural networks

An artificial neural network (ANN) learning algorithm, usually called "neural network" (NN), is
a learning algorithm that is vaguely inspired by biological neural networks. Computations are
structured in terms of an interconnected group of artificial neurons, processing information using
a connectionist approach to computation. Modern neural networks are non-linear statistical data
modeling tools. They are usually used to model complex relationships between inputs
andoutputs, to find patterns in data, or to capture the statistical structure in an unknown joint
probability distribution between observed variables.

4. Deep learning
Falling hardware prices and the development of GPUs for personal use in the last few years have
contributed to the development of the concept of deep learning which consists of multiple hidden
layers in an artificial neural network. This approach tries to model the way the human brain
processes light and sound into vision and hearing. Some successful applications of deep learning
are computer vision and speech Recognition.

5. Inductive logic programming

Inductive logic programming (ILP) is an approach to rule learning using logic Programming as a
uniform representation for input examples, background knowledge, and hypotheses. Given an
encoding of the known background knowledge and a set of examples represented as a logical
database of facts, an ILP system will derive a hypothesized logic program that entails all positive
and no negative examples. Inductive programming is a related field that considers any kind of
programming languages for representing hypotheses (and not only logic programming), such as
functional programs.
6. Support vector machines
Support vector machines (SVMs) are a set of related supervised learning methods used for
classification and regression. Given a set of training examples, each marked as belonging to one

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 5 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

of two categories, an SVM training algorithm builds a model that predicts whether a new
example falls into one category or the other.

7. Clustering
Cluster analysis is the assignment of a set of observations into subsets (called clusters) so that
observations within the same cluster are similar according to some pre designated criterion or
criteria, while observations drawn from different clusters are dissimilar. Different clustering
techniques make different assumptions on the structure of the data, often defined by some
similarity metric and evaluated for example by internal compactness (similarity between
members of the same cluster) and separation between different clusters. Other methods are
based on estimated density and graph connectivity. Clustering is a method of unsupervised
learning,and a common technique for statistical data analysis.

8. Bayesian networks
A Bayesian network, belief network or directed acyclic graphical model is a probabilistic
graphical model that represents a set of random variables and their conditional
independenciesvia a directed acyclic graph (DAG). For example, a Bayesian network could
represent the probabilistic relationships between diseases and symptoms. Given symptoms, the
network can be used to compute the probabilities of the presence of various diseases. Efficient
algorithms exist that perform inference and learning.

9. Reinforcement learning
Reinforcement learning is concerned with how an agent ought to take actions in an environment
so as to maximize some notion of long-term reward. Reinforcement learning algorithms attempt
to find a policy that maps states of the world to the actions the agent ought to take in those states.
Reinforcement learning differs from the supervised learning problem in that correct input/output
pairs are never presented, nor sub-optimal actions explicitly corrected.
10. Similarity and metric learning
In this problem, the learning machine is given pairs of examples that are considered similar and
pairs of less similar objects. It then needs to learn a similarity function (or a distance metric
function) that can predict if new objects are similar. It is sometimes used in Recommendation
systems.

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 6 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

11. Genetic algorithms

A genetic algorithm (GA) is a search heuristic that mimics the process of natural selection, and
uses methods such as mutation and crossover to generate new genotype in the hope of finding
good solutions to a given problem. In machine learning, genetic algorithms found some uses in
the 1980s and 1990s. Conversely, machine learning techniques have been used to improve the
performance of genetic and evolutionary algorithms.

12. Rule-based machine learning

Rule-based machine learning is a general term for any machine learning method that identifies,
learns, or evolves "rules" to store, manipulate or apply, knowledge. The defining characteristic of
a rule- based machine learner is the identification and utilization of a set of relational rules that
collectively represent the knowledge captured by the system. This is in contrast to other machine
learners that commonly identify a singular model that can be universally applied to any instance
in order to make a prediction. Rule-based machine learning approaches include learning
classifier systems, association rule learning, and artificial immune systems.

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 7 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

1. Apply and demonstrate the FIND-S algorithm for finding the most specific Hypothesis
based on a given set of training data samples.
Find-s Algorithm :
1.Load Data set
2. Initialize h to the most specific hypothesis in H
3.For each positive training instance x
• For each attribute constraint ai in h
If the constraint ai in h is satisfied by x then do nothing
else replace ai in h by the next more general constraint that is satisfied by x
4.Output hypothesis h

Source Code:
import random
import csv

def read_data(filename):
with open(filename, 'r') as csvfile:
datareader = csv.reader(csvfile, delimiter=',')
traindata = []
for row in datareader:
traindata.append(row)
return (traindata)

h=['phi','phi','phi','phi','phi','phi'
data=read_data('finds.csv')
def isConsistent(h,d):
if len(h)!=len(d)-1:
print('Number of attributes are not same in hypothesis.')
return False
else:
matched=0
for i in range(len(h)):
if ( (h[i]==d[i]) | (h[i]=='any')
): matched=matched+1
if matched==len(h):
return True
else:
return False
def makeConsistent(h,d):
for i in
range(len(h)):
if((h[i] == 'phi')):
h[i]=d[i]
elif(h[i]!=d[i]):
h[i]='any'
return h
print('Begin : Hypothesis :',h)

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 8 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

print('==========================================')
for d in data:
if d[len(d)-1]=='Yes':
if ( isConsistent(h,d)):
pass
else:
h=makeConsistent(h,d)
print ('Training data :',d)
print ('Updated Hypothesis :',h)
print()
print(' ')
print('==========================================')
print('maximally sepcific data set End: Hypothesis :',h)

Output:
Begin : Hypothesis : ['phi', 'phi', 'phi', 'phi', 'phi', 'phi']
==========================================
Training data : ['Cloudy', 'Cold', 'High', 'Strong', 'Warm', 'Change', 'Yes']
Updated Hypothesis : ['Cloudy', 'Cold', 'High', 'Strong', 'Warm', 'Change']

Training data : ['Sunny', 'Warm', 'Normal', 'Strong', 'Warm', 'Same', 'Yes']

Updated Hypothesis : ['any', 'any', 'any', 'Strong', 'Warm', 'any']

Training data : ['Sunny', 'Warm', 'High', 'Strong', 'Warm', 'Same', 'Yes']

Updated Hypothesis : ['any', 'any', 'any', 'Strong', 'Warm', 'any']

Training data : ['Sunny', 'Warm', 'High', 'Strong', 'Cool', 'Change', 'Yes']

Updated Hypothesis : ['any', 'any', 'any', 'Strong', 'any', 'any']

Training data : ['Overcast', 'Cool', 'Normal', 'Strong', 'Warm', 'Same', 'Yes']

Updated Hypothesis : ['any', 'any', 'any', 'Strong', 'any', 'any']

==========================================
maximally sepcific data set End: Hypothesis : ['any', 'any', 'any', 'Strong', 'any',

'any'] OR

import csv
def loadCsv(filename):
lines = csv.reader(open(filename, "r"))
dataset = list(lines)

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 9 / 57
 lOMoARcPSD|45333583

www.BrainKart.com
for i in
range(len(dataset)):
dataset[i] = dataset[i]

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 10 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

return dataset
attributes = ['Sky','Temp','Humidity','Wind','Water','Forecast']
print('Attributes =',attributes)
num_attributes = len(attributes)

filename = "finds.csv"
dataset = loadCsv(filename)
print(dataset)
hypothesis=['0'] * num_attributes
print("Intial Hypothesis")
print(hypothesis)
print("The Hypothesis are")
for i in range(len(dataset)):
target = dataset[i][-1]
if(target == 'Yes'):
for j in
range(num_attributes):
if(hypothesis[j]=='0'):
hypothesis[j] = dataset[i][j]
if(hypothesis[j]!= dataset[i][j]):
hypothesis[j]='?'
print(i+1,'=',hypothesis)
print("Final Hypothesis")
print(hypothesis)

Output:
Attributes = ['Sky', 'Temp', 'Humidity', 'Wind', 'Water', 'Forecast']
[['sky', 'Airtemp', 'Humidity', 'Wind', 'Water', 'Forecast', 'WaterSport'],
['Cloudy', 'Cold', 'High', 'Strong', 'Warm', 'Change', 'Yes'],
['Sunny', 'Warm', 'Normal', 'Strong', 'Warm', 'Same', 'Yes'],
['Sunny', 'Warm', 'High', 'Strong', 'Warm', 'Same', 'Yes'],
['Cloudy', 'Cold', 'High', 'Strong', 'Warm', 'Change', 'No'],
['Sunny', 'Warm', 'High', 'Strong', 'Cool', 'Change', 'Yes'],
['Rain', 'Mild', 'High', 'Weak', 'Cool', 'Change', 'No'],
['Rain', 'Cool', 'Normal', 'Weak', 'Cool', 'Same', 'No'],
['Overcast', 'Cool', 'Normal', 'Strong', 'Warm', 'Same', 'Yes']]
Intial Hypothesis
['0', '0', '0', '0', '0',
'0']
The Hypothesis are
2 = ['Cloudy', 'Cold', 'High', 'Strong', 'Warm', 'Change']
3 = ['?', '?', '?', 'Strong', 'Warm', '?']
4 = ['?', '?', '?', 'Strong', 'Warm', '?']
6 = ['?', '?', '?', 'Strong', '?', '?']
9 = ['?', '?', '?', 'Strong', '?', '?']
Final Hypothesis
['?', '?', '?', 'Strong', '?', '?']

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 11 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

2. Implement and demonstrate the Candidate-Elimination algorithm to output a description of the set of all
hypotheses consistent with the training examples.

Candidate-Elimination Algorithm:
1. Load data set
2. G <-maximally general hypotheses in H
3. S <- maximally specific hypotheses in H
4. For each training example
d=<x,c(x)>Case 1 : If d is a positive
example
Remove from G any hypothesis that is inconsistent with
dFor each hypothesis s in S that is not consistent with d
• Remove s from S.
• Add to S all minimal generalizations h of s such that
• h consistent with d
• Some member of G is more general than h
• Remove from S any hypothesis that is more general than another hypothesis in S

Case 2: If d is a negative example

Remove from S any hypothesis that is
inconsistent with dFor each hypothesis g in G
that is not consistent with d
*Remove g from G.
*Add to G all minimal specializations h of g such that
o h consistent with d
o Some member of S is more specific than h
• Remove from G any hypothesis that is less general than another hypothesis in G

Source Code:

import numpy as
np import pandas
as pd

data =
pd.DataFrame(data=pd.read_csv('finds1.csv'))
concepts = np.array(data.iloc[:,0:-1])

target = np.array(data.iloc[:,-1])
def learn(concepts, target):
specific_h = concepts[0].copy()
print("initialization of specific_h and
general_h") print(specific_h)
general_h = [["?" for i in range(len(specific_h))] for i in
range(len(specific_h))] print(general_h)

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 12 / 57
 lOMoARcPSD|45333583

www.BrainKart.com
for i, h in
enumerate(concepts): if
target[i] == "Yes":

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 13 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

for x in
range(len(specific_h)): if
h[x] != specific_h[x]:
specific_h[x] = '?'
general_h[x][x] = '?'
if target[i] == "No":
for x in
range(len(specific_h)): if
h[x] != specific_h[x]:
general_h[x][x] = specific_h[x]
else:
general_h[x][x] = '?'
print(" steps of Candidate Elimination
Algorithm",i+1) print("Specific_h ",i+1,"\n ")
print(specific_h)
print("general_h ", i+1, "\n ")
print(general_h)

indices = [i for i, val in enumerate(general_h) if val == ['?', '?', '?', '?', '?',
'?']] for i in indices:
general_h.remove(['?', '?', '?', '?', '?', '?'])

return specific_h, general_h

s_final, g_final = learn(concepts, target)
print("Final Specific_h:", s_final, sep="\n")
print("Final General_h:", g_final, sep="\n")

OUTPUT
initialization of specific_h and general_h
['Cloudy' 'Cold' 'High' 'Strong' 'Warm' 'Change']
[['?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?'], ['?', '?', '?',
'?', '?', '?'], ['?', '?', '?', '?', '?', '?']]
steps of Candidate Elimination Algorithm
8 Specific_h 8

['?' '?' '?' 'Strong' '?' '?']

general_h 8

[['?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?'], ['?', '?', '?', 'Strong', '?', '?'], ['?',
'?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?']]
Final Specific_h:
['?' '?' '?' 'Strong' '?' '?']
Final General_h:
[['?', '?', '?', 'Strong', '?', '?']]

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 14 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

3. Write a program to demonstrate the working of the decision tree based on ID3 algorithm. Use an appropriate
data set for building the decision tree and apply this knowledge to classify a new sample.

Outlook

sunny overcast rainy

Humidity yes Windy

normal false true

High

no yes yes no

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 15 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

Source Code:
import numpy as
np import math
from data_loader import read_data
class Node:
def init (self, attribute):
self.attribute = attribute
self.children = []
self.answer = ""
def str (self):
return self.attribute
def subtables(data, col,
delete): dict = {}
items = np.unique(data[:, col])
count = np.zeros((items.shape[0], 1), dtype=np.int32)
for x in range(items.shape[0]):
for y in
range(data.shape[0]):
if data[y, col] == items[x]:
count[x] += 1
for x in range(items.shape[0]):
dict[items[x]] = np.empty((int(count[x]), data.shape[1]), dtype="|
S32") pos = 0
for y in range(data.shape[0]):
if data[y, col] == items[x]:
dict[items[x]][pos] = data[y]
pos += 1
if delete:
dict[items[x]] = np.delete(dict[items[x]], col, 1)
return items, dict
def entropy(S):
items = np.unique(S)
if items.size ==
1: return 0
counts = np.zeros((items.shape[0], 1))
sums = 0
for x in range(items.shape[0]):
counts[x] = sum(S == items[x]) / (S.size * 1.0)
for count in counts:
sums += -1 * count * math.log(count, 2)
return sums
def gain_ratio(data, col):
items, dict = subtables(data, col, delete=False)
total_size = data.shape[0]
entropies = np.zeros((items.shape[0], 1))
intrinsic = np.zeros((items.shape[0], 1))
for x in range(items.shape[0]):
ratio = dict[items[x]].shape[0]/(total_size * 1.0)
entropies[x] = ratio * entropy(dict[items[x]][:, -1])
intrinsic[x] = ratio * math.log(ratio, 2)

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 16 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

total_entropy = entropy(data[:, -1])

iv = -1 * sum(intrinsic)
for x in range(entropies.shape[0]):
total_entropy -= entropies[x]
return total_entropy / iv
def create_node(data, metadata):
#TODO: Co jeśli information gain jest zerowe?
if (np.unique(data[:, -1])).shape[0] == 1:
node = Node("")
node.answer = np.unique(data[:, -1])[0]
return node
gains = np.zeros((data.shape[1] - 1, 1))
for col in range(data.shape[1] - 1):
gains[col] = gain_ratio(data, col)
split = np.argmax(gains)
node =
Node(metadata[split])
metadata = np.delete(metadata, split, 0)
items, dict = subtables(data, split,
delete=True) for x in range(items.shape[0]):
child = create_node(dict[items[x]], metadata)
node.children.append((items[x], child))
return node
def empty(size):
s = ""
for x in
range(size): s +=
" "
return s
def print_tree(node, level):
if node.answer != "":
print(empty(level), node.answer)
return
print(empty(level), node.attribute)
for value, n in node.children:
print(empty(level + 1), value)
print_tree(n, level + 2)

metadata, traindata =
read_data("tennis.data") data =
np.array(traindata)
node = create_node(data,
metadata) print_tree(node, 0)
OUTPUT:

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 17 / 57
 lOMoARcPSD|45333583

www.BrainKart.com
outlook
overcast
b'yes'
rain

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 18 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

wind
b'strong'
b'no'
b'weak'
b'yes'
sunny
humidity
b'high'
b'no'
b'normal'
b'yes'

OR
import pandas as
pd import numpy
as np
dataset= pd.read_csv('playtennis.csv',names=['outlook','temperature','humidity','wind','class',])
def entropy(target_col):
elements,counts = np.unique(target_col,return_counts = True)
entropy = np.sum([(-counts[i]/np.sum(counts))*np.log2(counts[i]/np.sum(counts)) for i in
range(len(elements))])
return entropy
def InfoGain(data,split_attribute_name,target_name="class"):
total_entropy = entropy(data[target_name])
vals,counts= np.unique(data[split_attribute_name],return_counts=True)
Weighted_Entropy =
np.sum([(counts[i]/np.sum(counts))*entropy(data.where(data[split_attribute_name]==vals[i]).dr opna()
[target_name]) for i in range(len(vals))])
Information_Gain = total_entropy - Weighted_Entropy
return Information_Gain
def ID3(data,originaldata,features,target_attribute_name="class",parent_node_class =
None): if len(np.unique(data[target_attribute_name])) <= 1:
return np.unique(data[target_attribute_name])[0]
elif len(data)==0:
return np.unique(originaldata[target_attribute_name])
[np.argmax(np.unique(originaldata[target_attribut e_name],return_counts=True)[1])]
elif len(features) ==0:
return parent_node_class
else:
parent_node_class =
np.unique(data[target_attribute_name])[np.argmax(np.unique(data[target_attribute_name],return
_counts=True)[1])]
item_values = [InfoGain(data,feature,target_attribute_name) for feature in features] #Return
the information gain values for the features in the dataset
best_feature_index = np.argmax(item_values)
best_feature = features[best_feature_index]
tree = {best_feature:{}}
features = [i for i in features if i !=
best_feature] for value in
np.unique(data[best_feature]):
value = value
sub_data = data.where(data[best_feature] == value).dropna()
subtree = ID3(sub_data,dataset,features,target_attribute_name,parent_node_class) tree[best_feature]
[value] = subtree
return(tree)
tree = ID3(dataset,dataset,dataset.columns[:-1])
print(' \nDisplay Tree\n',tree)
OUTPUT:
Display Tree
{'outlook': {'Overcast': 'Yes', 'Rain': {'wind': {'Strong': 'No', 'Weak': 'Yes'}}, 'Sunny':

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 19 / 57
 lOMoARcPSD|45333583

www.BrainKart.com
{'humidity': {'High': 'No', 'Normal': 'Yes'}}}}4.

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 20 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

Task 4: Write a program to implement the Gaussian mixture model. Compute the accuracy of the classifier,
considering few test data sets.

Algorithm

1. Initialize the mean μk, the covariance matrix k, and the mixing coefficients k by some random
variables.
2. Compute the k
3. Again, estimate all the parameters using the current k values
4. Compute log-likelihood function.
5. Put some convergence criterion
6. If the log-likelihood value converges to some value ( or if all the parameters converge to some values
) then stop, else return to Step 2.

Python Code
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from pandas import DataFrame
from sklearn import datasets
from sklearn.mixture import GaussianMixture

# load the iris dataset

iris = datasets.load_iris()
# select first two columns
X = iris.data[:, :2]
# turn it into a dataframe
d = pd.DataFrame(X)
# plot the data
plt.scatter(d[0], d[1])
gmm = GaussianMixture(n_components = 3)
# Fit the GMM model for the dataset
# which expresses the dataset as a
# mixture of 3 Gaussian Distribution
gmm.fit(d)
# Assign a label to each sample
labels = gmm.predict(d)
d['labels']= labels
d0 = d[d['labels']== 0]
d1 = d[d['labels']== 1]
d2 = d[d['labels']== 2]
plt.show()

# plot three clusters in same plot

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 21 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

plt.scatter(d0[0], d0[1], c ='r')

plt.scatter(d1[0], d1[1], c ='yellow')
plt.scatter(d2[0], d2[1], c ='g')
plt.show()

# print the converged log-likelihood value

print("The mean:",gmm.means_)
print('\n')
print("The covariance matrix:",gmm.covariances_)
print("The converged log-likelihood value:",gmm.lower_bound_)
# print the number of iterations needed
# for the log-likelihood value to converge
print("The number of iterations needed for the log-likelihood value to converge:",gmm.n_iter_)

output:

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 22 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

The mean: [[5.9009976 2.74387546]

[6.68055626 3.02849627]
[5.01507898 3.4514463 ]]

The covariance matrix: [[[0.27671149 0.08897036]

[0.08897036 0.09389206]]

[[0.36153508 0.05159664]
[0.05159664 0.08927917]]

[[0.11944714 0.08835648]
[0.08835648 0.11893388]]]
The converged log-likelihood value: -1.4987505566235166
The number of iterations needed for the log-likelihood value to converge: 8

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 23 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

5. Buildan Artificial Neural Network by implementing the Back propagation Algorithm

and test the same using appropriate data sets.

Back propagation Algorithm:

1.Load data set
2. Assign all network inputs and output
3.Initialize all weights with small random numbers, typically between -1 and 1
repeat
for every pattern in the training set
Present the pattern to the
network
// Propagated the input forward through the
network: for each layer in the network
for every node in the layer
1. Calculate the weight sum of the inputs to the node
2. Add the threshold to the sum
3. Calculate the activation for the
node end
end

// Propagate the errors backward through the

network for every node in the output layer
calculate the error
signal end

for all hidden layers

for every node in the layer

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 24 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

1. Calculate the node's signal error

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 25 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

2. Update each node's weight in the

network end
end

// Calculate Global Error

Calculate the Error
Function

end

while ((maximum number of iterations < than specified) AND

(Error Function is > than specified))

 Input layer with two inputs neurons

 One hidden layer with two neurons
 Output layer with a single neuron
Source Code:

import numpy as np
X = np.array(([2, 9], [1, 5], [3, 6]), dtype=float)
y = np.array(([92], [86], [89]), dtype=float)
X = X/np.amax(X,axis=0) # maximum of X array longitudinally y = y/100

#Sigmoid Function
def sigmoid (x):
return (1/(1 + np.exp(-x)))
#Derivative of Sigmoid
Function def
derivatives_sigmoid(x):
return x * (1 - x)
#Variable initialization
epoch=7000 #Setting training iterations

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 26 / 57
 lOMoARcPSD|45333583

www.BrainKart.com
lr=0.1 #Setting learning rate
inputlayer_neurons = 2 #number of features in data set

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 27 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

hiddenlayer_neurons = 3 #number of hidden layers neurons

output_neurons = 1 #number of neurons at output
layer
#weight and bias initialization
wh=np.random.uniform(size=(inputlayer_neurons,hiddenlayer_neurons))
bh=np.random.uniform(size=(1,hiddenlayer_neurons))
wout=np.random.uniform(size=(hiddenlayer_neurons,output_neurons))

bout=np.random.uniform(size=(1,output_neurons))
# draws a random range of numbers uniformly of dim x*y
#Forward Propagation
for i in range(epoch):
hinp1=np.dot(X,wh)
hinp=hinp1 + bh
hlayer_act =
sigmoid(hinp)
outinp1=np.dot(hlayer_act,wout)
outinp= outinp1+ bout
output = sigmoid(outinp)
#Backpropagation
EO = y-output
outgrad =
derivatives_sigmoid(output) d_output
= EO* outgrad
EH = d_output.dot(wout.T)
hiddengrad = derivatives_sigmoid(hlayer_act)
#how much hidden layer wts contributed to error
d_hiddenlayer = EH * hiddengrad
wout += hlayer_act.T.dot(d_output) *lr
# dotproduct of nextlayererror and currentlayerop
bout += np.sum(d_output, axis=0,keepdims=True)
*lr wh += X.T.dot(d_hiddenlayer) *lr
#bh += np.sum(d_hiddenlayer, axis=0,keepdims=True) *lr
print("Input: \n" + str(X))
print("Actual Output: \n" + str(y))
print("Predicted Output: \n"
,output)

Output:
Input:
[[ 0.66666667 1. ]
[ 0.33333333 0.55555556]
[ 1. 0.66666667]]

Actual Output:
[[ 0.92]
[ 0.86]
[ 0.89]]

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 28 / 57
 lOMoARcPSD|45333583

www.BrainKart.com
Predicted Output:
[[ 0.89559591]
[ 0.88142069]
[ 0.8928407 ]]

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 29 / 57
 lOMoARcPSD|45333583

www.BrainKart.com
ML LAB

6. Task 6: Assuming a set of documents that need to be classified, use the Radial basis function Classifier
model to perform this task. Calculate the classification rate, accuracy, precision, and recall for your data
set.

Methodology
Radial basis function network (or RBFN for short) is an artificial neural network that uses radial basis
functions as activation functions. The output of the network is a linear combination of radial basis
functions of the inputs and neuron parameters.
Requirements:

 Numpy and Matplotlib

Extract the training data inside the Network.py directory. Running the Network.py script, you get two
options: Train and Predict.

Train

Training will import the 60,000 training examples and train using that data. Finally the weights and
centers (used for RBF) will be saved.
By default, centers/centriods for the RBF neurons are selected as the first 300 training examples. K-
means cluserting can also be used to get centers by setting useKMeans to True. If kMeans is used then
remember to lower the numbers of centers from 300 otherwise training will take considerably long time.

Predict

Predict will import the 10,000 test examples and the weights (weights.npy). Using the weights, the
Network will predict results and compare and finally compute the classification accuracy of the Network.

Learning Curves

For diagnostics, other than classification accuracy, you can plot the learning curves to check how the
Network is learning. The function plotLearningCurves is included and can be called after predicting.

Training Data

Training Data used from MNIST Database. 60,000 training examples and 10,000 test examples.
The data is in text files and zipped (compressed file size: 20.6 Mb).

Network Structure

Architecture

Size: [1, 300, 10]

 1 input layer neurons

Each training example is an image of 784 pixels (28 x 28) so there are 784 features for the input
layer. However, since this is an RBF Network, instead of having a neuron for each feature, we
Dept. of CSE, Page 35

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 30 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

have 1 neuron which propogates each training example of 784 features to the hidden layer RBF
Neurons

 300 hidden layer neurons

Depends on the number of centers C (or clusters K) chosen

 10 output layer neurons

Each for classifying a digit 0-9

RBF activation function

The RBF function used is a Gaussian function as shown:

The centers c are selected based on random sampling of 300 data samples.
Centers can also be chosen using the k-means clustering algorithm available in the file. However, by
default, using k-means cluserting has been set to False, and instead we use the random sampling.

Backpropogation

Backpropogation uses stochastic gradient descent to update weights connecting hidden layer to output
layer.

Performance

The parameters used for this performance test are as below. These parameters have been fixed (hard-
coded) as of now, but can be changed as you wish from the Network.py code.

Parameters Value
beta 0.05
Learning rate 0.3
Num of Epochs 1
Training using all 60,000 examples takes around 3-4 minutes. Code

import
numpy
as np
import time
import re
import sys
import math
import matplotlib.pyplot as plt

np.set_printoptions(threshold=sys.maxsize, suppress=True)
np.random.seed(1)

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 31 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

def inputXFromFile(filename, sampleSize): # SampleSize given for performace

enhancement
"""Inputs the training examples X"""
inputArray = np.zeros(shape=(sampleSize, 784)) # 784 = pixels of each image
with open(filename, "r") as file:
for i in range(sampleSize):
inputList = []
for _ in range(44): # 44 lines of each example in file
line = file.readline().strip("[").replace("]", "")
inputList += line.split()
inputArray[i] = inputList
# print("X Input Size:", inputArray.shape)
return np.divide(inputArray, 255)

def inputYFromFile(filename, sampleSize):

"""Inputs the training examples Y"""
# for each row, we want a column like [0 0 1 0 ...]
inputArray = np.zeros(shape=(sampleSize, 10))
with open(filename, "r") as file:
for i in range(sampleSize):
value = file.readline()
if not value:
break
inputArray[i][int(value)] = 1
# print("Y input size:", inputArray.shape)
return inputArray

def kMeansClustering(K, sampleData):

"""Find K cluster centeriods of the passed sampleData
"""
# Randomly select K examples as starting point for centeriods
randIndices = np.random.choice(sampleData.shape[0], size=K, replace=False)
centeriods = sampleData[randIndices, :]

dataSize = 10000
data = sampleData[:dataSize]
for i in range(15):
centeriodSums = np.zeros(shape=centeriods.shape)
centeriodSumsCounter = np.zeros(shape=K)

# 1. Allocate a training example to its nearest centriod

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 32 / 57
 lOMoARcPSD|45333583

www.BrainKart.com
for x in data:
# Get index of closest centeriod to X (minimum of ||C - X||)
index = np.argmin(np.square(centeriods - x).sum(axis=1))
centeriodSums[index] += x
centeriodSumsCounter[index] += 1

# 2. Reassign the centriods to new means

for i in range(K):
centeriods[i] = centeriodSums[i] / centeriodSumsCounter[i]

return centeriods

class Network:
def init (self):
self.XSize = 0
self.HSize = 300
self.OSize = 10
self.X = []
self.C = []
self.Y = []
self.W = []
# self.B = np.random.uniform(-1, 1, (self.OSize))
self.trainErrors = []
self.testErrors = []

def loadData(self, filenameX, filenameY, sampleSize):

"""Loads training/test data\n
Parameters:\n
- filenameX: filename for X features\n
- filenameY: filename for Y (labels)\n
- sampleSize: number of examples in dataset
"""
self.X = inputXFromFile(filenameX, sampleSize)
self.Y = inputYFromFile(filenameY, sampleSize)
self.XSize = sampleSize

def initializeCenters(self, K, useKMeans):

"""Initializes Centers (for RBF neurons in hidden layer)\n
Parameters:\n
- useKMeans: Set to true to use kMeans clustering to get centroids
"""
print("Initialzing Centers...")
self.HSize = K # Since centriods is equal to hidden layer neurons
if useKMeans:
self.C = kMeansClustering(K, self.X)

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 33 / 57
 lOMoARcPSD|45333583

www.BrainKart.com
else:
self.C = self.X[: self.HSize]

def train(self, epochs=1, learnRate=0.5, K=300, useKMeans=False):

"""Trains the Network\n
Parameters:\n
- epochs: Number of epochs or trainings on entire dataset
- learnRate: Learning Rate
- K: Number of centers/centroids
- useKMeans: Set to true to use kMeans clustering to get centroids
"""
# Initialzing centers and weights
self.initializeCenters(K, useKMeans)
self.W = np.random.uniform(-1, 1, (self.HSize, self.OSize))

self.trainErrors = np.zeros(shape=self.XSize) # Preallocating numpy array

print("Training...")
for _ in range(epochs):
# Take each data sample from the inputData
for i, x in enumerate(self.X):
HLayer = rbf(x, self.C)
# Multiply the weights to get output for each data
output = np.dot(HLayer, self.W) # + self.B
error = output - self.Y[i]
self.W = self.W - (learnRate * np.outer(HLayer, error))
# self.B = self.B - (learnRate * error)
self.trainErrors[i] = 0.5 * sum(error ** 2)
print("Training done")
# Saving weights and centers in a file
np.save("weights", self.W)
np.save("centers", self.C)

def predict(self):
"""Predicts using the Network by the set parameters (weight and
centers) and displays the accuracy
"""
self.testErrors = np.zeros(shape=self.XSize) # Preallocating numpy array
print("Prediciting...")
totalAvg = count = correctCount = 0.0
# Take each data sample from the inputData
for count, x in enumerate(self.X):
HLayer = rbf(x, self.C)
output = np.dot(HLayer, self.W) # + self.B
o = np.argmax(output)
y = np.argmax(self.Y[count])

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 34 / 57
 lOMoARcPSD|45333583

www.BrainKart.com
if o == y:
correctCount += 1

error = output - self.Y[count]

self.testErrors[count] = 0.5 * sum(error ** 2)

totalAvg = (correctCount * 100.0) / (count + 1)

print(f"Total Avg. Accuracy: {totalAvg} %")

def rbf(x, C, beta=0.05):

"""Radial Basis Function\n
Parameters:\n
x: a training example
C: centers of used for the hidden layer
"""
H = np.zeros(shape=(np.shape(C)[0]))
for i, c in enumerate(C): # For each neuron in H layer
H[i] = math.exp((-1 * beta) * np.dot(x - c, x - c))
return H

def plotLearningCurves(trainErrors, testErrors):

"""Plots the learning curves of both training cost and test cost
"""
# Averaging over the first {avgSize} examples
avgSize = 100
if type(trainErrors) is np.ndarray: # if trainError data is available
Jtrain = trainErrors.reshape(-1, avgSize).mean(axis=1)
plt.plot(Jtrain, label='Training Cost')
Jtest = testErrors.reshape(-1, avgSize).mean(axis=1)
plt.plot(Jtest, label='Test Cost')
plt.xlabel(f"Data examples in {avgSize}s")
plt.ylabel("Cost")
plt.show()

####### MAIN ######

if name == " main ":
start = time.time()
trainDataSize = 60000
testDataSize = 10000

myNetwork = Network()
while True:

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 35 / 57
 lOMoARcPSD|45333583

www.BrainKart.com
ML LAB
print("1. Train the RBF Network\n2. Predict using the RBF Network")
userInput = input("Choose your option: ")
if userInput == "1":
print("Importing data for training...")
startTime = time.time()
myNetwork.loadData("train.txt", "train-labels.txt", trainDataSize)
print(
f"{trainDataSize} training examples imported in {time.time()-startTime:.2f} sec"
)

startTrainingTime = time.time()
myNetwork.train(epochs=1, learnRate=0.3, K=300, useKMeans=False)
print(f"Training took: {time.time()-startTrainingTime:.2f} sec")
elif userInput == "2":
# Loading centers and weights from save file
filename = input("Enter file name containing weights (default: weights.npy): ")
myNetwork.W = np.load(filename)
myNetwork.C = np.load("centers.npy")

print("Importing data for testing...")

myNetwork.loadData("test.txt", "test-labels.txt", testDataSize)
myNetwork.predict()
# Plotting the errors of the first 10000 examples
plotLearningCurves(myNetwork.trainErrors[:10000],
myNetwork.testErrors[:10000])
else:
break
print("Program exited.")

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 36 / 57
 lOMoARcPSD|45333583

www.BrainKart.com
ML LAB

7. Write a program to construct a Hidden markov model for a medical data. Use this model to demonstrate
the diagnosis of medial data.

Methodology

Hidden Markov models are used to ferret out the underlying, or hidden, sequence of states that generates
a set of observations. In his now canonical toy example, Jason Eisner uses a series of daily ice cream
consumption (1, 2, 3) to understand Baltimore's weather for a given summer (Hot/Cold days). These are
arrived at using transmission probabilities (i.e. the likelihood of moving from one state to another) and
emission probabilities (i.e. the likelihood of seeing a particular observation given an underlying state).

This implementation adopts his approach into a system that can take:

 An initial transmission matrix

 An initial emission matrix
 A set of observations

You can see an example input by using the main() function call on the hmm.py file.
HMM models calculate first the probability of a given sequence and its individual observations for
possible hidden state sequences, then re-calculate the matrices above given those probabilities. By
iterating back and forth (what's called an expectation-maximization process), the model arrives at a local
optimum for the tranmission and emission probabilities. It's a pretty good outcome for what might
otherwise be a very hefty computationally difficult problem.

This model implements the forward-backward algorithm recursively for probability calculation within the
broader expectation-maximization pattern.

Code

# A class for performing hidden markov models

import copy
import numpy as np

class HMM():

def init (self, transmission_prob, emission_prob, obs=None):

'''
Note that this implementation assumes that n, m, and T are small
enough not to require underflow mitigation.
Required Inputs:
- transmission_prob: an (n+2) x (n+2) numpy array, initial, where n is
the number of hidden states
Dept. of CSE, Page 46

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 37 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

- emission_prob: an (m x n) 2-D numpy array, where m is the number of

possible observations
Optional Input:
- obs: a list of observation labels, in the same order as their
occurence within the emission probability matrix; otherwise, will assume
that the emission probabilities are in alpha-numerical order.
'''
self.transmission_prob = transmission_prob
self.emission_prob = emission_prob
self.n = self.emission_prob.shape[1]
self.m = self.emission_prob.shape[0]
self.observations = None
self.forward = []
self.backward = []
self.psi = []
self.obs = obs
self.emiss_ref = {}
self.forward_final = [0 , 0]
self.backward_final = [0 , 0]
self.state_probs = []
if obs is None and self.observations is not None:
self.obs = self.assume_obs()

def assume_obs(self):
'''
If observation labels are not given, will assume that the emission
probabilities are in alpha-numerical order.
'''
obs = list(set(list(self.observations)))
obs.sort()
for i in range(len(obs)):
self.emiss_ref[obs[i]] = i
return obs

def train(self, observations, iterations = 10, verbose=True):

'''
Trains the model parameters according to the observation sequence.
Input:
- observations: 1-D string array of T observations
'''
self.observations = observations
self.obs = self.assume_obs()
self.psi = [[[0.0] * (len(self.observations)-1) for i in range(self.n)] for i in range(self.n)]
self.gamma = [[0.0] * (len(self.observations)) for i in range(self.n)]
for i in range(iterations):
old_transmission = self.transmission_prob.copy()
old_emission = self.emission_prob.copy()

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 38 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

if verbose:
print("Iteration: {}".format(i + 1))
self.expectation()
self.maximization()

def expectation(self):
'''
Executes expectation step.
'''
self.forward = self.forward_recurse(len(self.observations))
self.backward = self.backward_recurse(0)
self.get_gamma()
self.get_psi()

def get_gamma(self):
'''
Calculates the gamma matrix.
'''
self.gamma = [[0, 0] for i in range(len(self.observations))]
for i in range(len(self.observations)):
self.gamma[i][0] = (float(self.forward[0][i] * self.backward[0][i]) /
float(self.forward[0][i] * self.backward[0][i] +
self.forward[1][i] * self.backward[1][i]))
self.gamma[i][1] = (float(self.forward[1][i] * self.backward[1][i]) /
float(self.forward[0][i] * self.backward[0][i] +
self.forward[1][i] * self.backward[1][i]))

def get_psi(self):
'''
Runs the psi calculation.
'''
for t in range(1, len(self.observations)):
for j in range(self.n):
for i in range(self.n):
self.psi[i][j][t-1] = self.calculate_psi(t, i, j)

def calculate_psi(self, t, i, j):

'''
Calculates the psi for a transition from i->j for t > 0.
'''
alpha_tminus1_i = self.forward[i][t-1]
a_i_j = self.transmission_prob[j+1][i+1]
beta_t_j = self.backward[j][t]
observation = self.observations[t]
b_j = self.emission_prob[self.emiss_ref[observation]][j]
denom = float(self.forward[0][i] * self.backward[0][i] + self.forward[1][i] * self.backward[1][i])
return (alpha_tminus1_i * a_i_j * beta_t_j * b_j) / denom

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 39 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

def maximization(self):
'''
Executes maximization step.
'''
self.get_state_probs()
for i in range(self.n):
self.transmission_prob[i+1][0] = self.gamma[0][i]
self.transmission_prob[-1][i+1] = self.gamma[-1][i] / self.state_probs[i]
for j in range(self.n):
self.transmission_prob[j+1][i+1] = self.estimate_transmission(i, j)
for obs in range(self.m):
self.emission_prob[obs][i] = self.estimate_emission(i, obs)

def get_state_probs(self):
'''
Calculates total probability of a given state.
'''
self.state_probs = [0] * self.n
for state in range(self.n):
summ = 0
for row in self.gamma:
summ += row[state]
self.state_probs[state] = summ

def estimate_transmission(self, i, j):

'''
Estimates transmission probabilities from i to j.
'''
return sum(self.psi[i][j]) / self.state_probs[i]

def estimate_emission(self, j, observation):

'''
Estimate emission probability for an observation from state j.
'''
observation = self.obs[observation]
ts = [i for i in range(len(self.observations)) if self.observations[i] == observation]
for i in range(len(ts)):
ts[i] = self.gamma[ts[i]][j]
return sum(ts) / self.state_probs[j]

def backward_recurse(self, index):

'''
Runs the backward recursion.
'''
# Initialization at T
if index == (len(self.observations) - 1):

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 40 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

backward = [[0.0] * (len(self.observations)) for i in range(self.n)]

for state in range(self.n):
backward[state][index] = self.backward_initial(state)
return backward
# Recursion for T --> 0
else:
backward = self.backward_recurse(index+1)
for state in range(self.n):
if index >= 0:
backward[state][index] = self.backward_probability(index, backward, state)
if index == 0:
self.backward_final[state] = self.backward_probability(index, backward, 0, final=True)
return backward

def backward_initial(self, state):

'''
Initialization of backward probabilities.
'''
return self.transmission_prob[self.n + 1][state + 1]

def backward_probability(self, index, backward, state, final=False):

'''
Calculates the backward probability at index = t.
'''
p = [0] * self.n
for j in range(self.n):
observation = self.observations[index + 1]
if not final:
a = self.transmission_prob[j + 1][state + 1]
else:
a = self.transmission_prob[j + 1][0]
b = self.emission_prob[self.emiss_ref[observation]][j]
beta = backward[j][index + 1]
p[j] = a * b * beta
return sum(p)

def forward_recurse(self, index):

'''
Executes forward recursion.
'''
# Initialization
if index == 0:
forward = [[0.0] * (len(self.observations)) for i in range(self.n)]
for state in range(self.n):
forward[state][index] = self.forward_initial(self.observations[index], state)
return forward
# Recursion

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 41 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

else:
forward = self.forward_recurse(index-1)
for state in range(self.n):
if index != len(self.observations):
forward[state][index] = self.forward_probability(index, forward, state)
else:
# Termination
self.forward_final[state] = self.forward_probability(index, forward, state, final=True)
return forward

def forward_initial(self, observation, state):

'''
Calculates initial forward probabilities.
'''
self.transmission_prob[state + 1][0] self.emission_prob[self.emiss_ref[observation]]
[state]
return self.transmission_prob[state + 1][0] * self.emission_prob[self.emiss_ref[observation]][state]

def forward_probability(self, index, forward, state, final=False):

'''
Calculates the alpha for index = t.
'''
p = [0] * self.n
for prev_state in range(self.n):
if not final:
# Recursion
obs_index = self.emiss_ref[self.observations[index]]
p[prev_state] = forward[prev_state][index-
1] * self.transmission_prob[state + 1][prev_state + 1] * self.emission_prob[obs_index][state]
else:
# Termination
p[prev_state] = forward[prev_state][index-1] * self.transmission_prob[self.n][prev_state + 1]
return sum(p)

def likelihood(self, new_observations):

'''
Returns the probability of a observation sequence based on current model
parameters.
'''
new_hmm = HMM(self.transmission_prob, self.emission_prob)
new_hmm.observations = new_observations
new_hmm.obs = new_hmm.assume_obs()
forward = new_hmm.forward_recurse(len(new_observations))
return sum(new_hmm.forward_final)

if name == ' main ':

# Example inputs from Jason Eisner's Ice Cream and Baltimore Summer example

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 42 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

# http://www.cs.jhu.edu/~jason/papers/#eisner-2002-tnlp
emission = np.array([[0.7, 0], [0.2, 0.3], [0.1, 0.7]])
transmission = np.array([ [0, 0, 0, 0], [0.5, 0.8, 0.2, 0], [0.5, 0.1, 0.7, 0], [0, 0.1, 0.1, 0]])
observations = ['2','3','3','2','3','2','3','2','2','3','1','3','3','1','1',
'1','2','1','1','1','3','1','2','1','1','1','2','3','3','2',
'3','2','2']
model = HMM(transmission, emission)
model.train(observations)
print("Model transmission probabilities:\n{}".format(model.transmission_prob))
print("Model emission probabilities:\n{}".format(model.emission_prob))
# Probability of a new sequence
new_seq = ['1', '2', '3']
print("Finding likelihood for {}".format(new_seq))
likelihood = model.likelihood(new_seq)
print("Likelihood: {}".format(likelihood))

Output:
Iteration: 1
Iteration: 2
Iteration: 3
Iteration: 4
Iteration: 5
Iteration: 6
Iteration: 7
Iteration: 8
Iteration: 9
Iteration: 10
Model transmission probabilities:
[[0.00000000e+00 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[1.44069481e-13 9.33776929e-01 7.18678407e-02 0.00000000e+00]
[1.00000000e+00 6.62230707e-02 8.64943107e-01 0.00000000e+00]
[0.00000000e+00 3.10801009e-14 6.31890522e-02 0.00000000e+00]]
Model emission probabilities:
[[0.64048542 0. ]
[0.14806851 0.5343899 ]
[0.21144608 0.4656101 ]]
Finding likelihood for ['1', '2', '3']
Likelihood: 3.2956914388507033e-15

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 43 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

8. Apply EM algorithm to cluster a set of data. Use the same data set for clustering using K- Means
algorithm. Compare the results of these two algorithms and comment on the quality of clustering. You
can add Java/Python ML library classes/API in the program.

Expectation-Maximization (EM) algorithm

Step 1: An initial guess is made for the model‟s parameters and a probability
distribution is created. This is sometimes called the “E-Step” for the
“Expected” distribution.
Step 2: Newly observed data is fed into the model.
Step 3: The probability distribution from the E-step is drawn to include the new
data. This is sometimes called the “M-step.”
Step 4: Steps 2 through 4 are repeated untilstability.

K- Means Algorithm
Step-1: Select the number K to decide the number of clusters.
Step-2: Select random K points or centroids. (It can be other from the input dataset).
Step-3: Assign each data point to their closest centroid, which will form the predefined K
clusters. Step-4: Calculate the variance and place a new centroid of each cluster.
Step-5: Repeat the third steps, which means reassign each datapoint to the new closest centroid of each
cluster.
Step-6: If any reassignment occurs, then go to step-4 else go to FINISH.
Step-7: The model is ready.

Code
import matplotlib.pyplot as plt
from sklearn import datasets
from sklearn.cluster import KMeans
import sklearn.metrics as sm
import pandas as pd
import numpy as np

iris = datasets.load_iris()

X = pd.DataFrame(iris.data)
X.columns = ['Sepal_Length','Sepal_Width','Petal_Length','Petal_Width']

y = pd.DataFrame(iris.target)
y.columns = ['Targets']

model = KMeans(n_clusters=3)
model.fit(X)

plt.figure(figsize=(14,7))

colormap = np.array(['red', 'lime', 'black'])

# Plot the Original Classifications

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 44 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

plt.subplot(1, 2, 1)
plt.scatter(X.Petal_Length, X.Petal_Width, c=colormap[y.Targets], s=40)
plt.title('Real Classification')
plt.xlabel('Petal Length')
plt.ylabel('Petal Width')

# Plot the Models Classifications

plt.subplot(1, 2, 2)
plt.scatter(X.Petal_Length, X.Petal_Width, c=colormap[model.labels_], s=40)
plt.title('K Mean Classification')
plt.xlabel('Petal Length')
plt.ylabel('Petal Width')
print('The accuracy score of K-Mean: ',sm.accuracy_score(y, model.labels_))
print('The Confusion matrixof K-Mean: ',sm.confusion_matrix(y, model.labels_))

from sklearn import preprocessing

scaler = preprocessing.StandardScaler()
scaler.fit(X)
xsa = scaler.transform(X)
xs = pd.DataFrame(xsa, columns = X.columns)
#xs.sample(5)

from sklearn.mixture import GaussianMixture

gmm = GaussianMixture(n_components=3)
gmm.fit(xs)

y_gmm = gmm.predict(xs)
#y_cluster_gmm

plt.subplot(2, 2, 3)
plt.scatter(X.Petal_Length, X.Petal_Width, c=colormap[y_gmm], s=40)
plt.title('GMM Classification')
plt.xlabel('Petal Length')
plt.ylabel('Petal Width')

print('The accuracy score of EM: ',sm.accuracy_score(y, y_gmm))

print('The Confusion matrix of EM: ',sm.confusion_matrix(y, y_gmm))

Output

The accuracy score of K-Mean: 0.8933333333333333

The Confusion matrixof K-Mean: [[50 0 0]
[ 0 48 2]
[ 0 14 36]]
The accuracy score of EM: 0.9666666666666667
The Confusion matrix of EM: [[50 0 0]

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 45 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

[ 0 45 5]
[ 0 0 50]]

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 46 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

Taks 9:
Apply LDA algorithm to select the appropriate data from the given data set. Use XG boost algorithm for
classification.

Algorithm

1. Compute the within class and between class scatter matrices

2. Compute the eigenvectors and corresponding eigenvalues for the scatter matrices

3. Sort the eigenvalues and select the top k

4. Create a new matrix containing eigenvectors that map to the k eigenvalues

5. Obtain the new features (i.e. LDA components) by taking the dot product of the data and
the matrix from step 4

Code

# In[ ]:

import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
from numpy import linalg as lg
get_ipython().magic(u'matplotlib inline')

df =
pd.read_csv('C:/Users/vyoms/Desktop/SCLC_study_output_filtered_2.csv',header
=None)

df1 = df.drop(df.index[0])

df2 = df1.drop(df.columns[0], axis=1)

df3 = df2

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 47 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

df3_1 = df2.values[0:20,:]
df3_2 = df2.values[20:, : ]

m_1 = df3_1.mean(axis = 0)
m_2 = df3_2.mean(axis = 0)
mean_all = df2.mean(axis = 0)

mean_1 = m_1.reshape(1,19)
mean_1 = np.repeat(mean_1,20,axis = 0)

mean_2 = m_2.reshape(1,19)
mean_2 = np.repeat(mean_2,20,axis = 0)

within_class_scatter = np.zeros((19,19))
wcs_1 = np.zeros((19,19))
wcs_1 = np.matmul((np.transpose(df3_1 - mean_1 )), (df3_1 - mean_1))

wcs_2 = np.zeros((19,19))
wcs_2 = np.matmul((np.transpose(df3_2 - mean_2 )), (df3_2 - mean_2))

within_class_scatter = np.add(wcs_1,wcs_2)

bcs_1 = np.multiply(len(df3_1),np.outer((m_1 - mean_all),(m_1 - mean_all)))

bcs_2 = np.multiply(len(df3_2),np.outer((m_2 - mean_all),(m_2 - mean_all)))

between_class_scatter = np.add(bcs_1,bcs_2)

e_val, e_vector =
np.linalg.eig(np.dot(lg.inv(within_class_scatter),between_class_scatter))
for e in range (len(e_val)):
e_scatter = e_vector[:,e].reshape(19,1)

print(e_val[e].real)

print(between_class_scatter)

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 48 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

eig_pairs = [(np.abs(e_val[i]).real, e_vector[:,i].real) for i in range(len(e_val))]

eig_pairs = sorted(eig_pairs, key=lambda k: k[0], reverse=True)

print('Eigenvalues in decreasing order:\n')

for i in eig_pairs:
print(i[0])

W= eig_pairs[0][1].reshape(19,1)

lda_project = np.dot(df2,W)

lda_project

# In[177]:

#plot
fig = plt.figure()
ax = fig.add_subplot(1, 1, 1)
ax.set_title('LDA')
ax.plot(lda_project[0:20], np.zeros(20), linestyle='None', marker='o', color='blue',
label='NSCLC')
ax.plot(lda_project[20:40], np.zeros(20), linestyle='None', marker='o', color='red',
label='SCLC')
fig.show()

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 49 / 57
 lOMoARcPSD|45333583

www.BrainKart.com
ML LAB
# In[185]:

from sklearn.discriminant_analysis import LinearDiscriminantAnalysis as LDA

y1_ = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2]

# LDA

sklearn_lda = LDA(n_components=1)
X_lda_sklearn = sklearn_lda.fit_transform(df2, y1_)

X_lda_sklearn= -X_lda_sklearn

print(X_lda_sklearn)

#plot
fig = plt.figure()
ax = fig.add_subplot(1, 1, 1)
ax.set_title('LDA')
ax.plot(X_lda_sklearn[0:20], np.zeros(20), linestyle='None', marker='o',
color='blue', label='NSCLC')
ax.plot(X_lda_sklearn[20:40], np.zeros(20), linestyle='None', marker='o',
color='red', label='SCLC')
fig.show()

Output

[ 3.15378176]
[ 4.86686796]
[ 2.81120157]
[ 3.93543558]
[ 3.39771836]
[ 3.25628819]

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 50 / 57
 lOMoARcPSD|45333583

www.BrainKart.com
ML LAB
[-4.49065733]
[-4.6115194 ]
[-3.45215203]
[-2.75643608]
[-3.83408221]
[-3.54607243]
[-3.25235288]
[-3.40306303]
[-4.04197759]
[-4.35872643]
[-5.76347876]
[-5.589837 ]
[-5.74304609]
[-3.85936631]
[-3.13967526]
[-3.56371641]
[-5.68605839]
[-4.9237813 ]
[-3.53641015]
[-4.242908 ]]

Dept. of CSE, Page 61

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 51 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 52 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

Task 10
Build a classification model that can effectively analyze and extract features from an image. Apply PCA
algorithm to find the appropriate feature.
Algorithm
Step 1: Standardize the dataset.
Step 2: Calculate the covariance matrix for the features in the dataset.
Step 3: Calculate the eigenvalues and eigenvectors for the covariance matrix.
Step 4: Sort eigenvalues and their corresponding eigenvectors.
Step 5: Pick k eigenvalues and form a matrix of eigenvectors.
Step 6: Transform the original matrix.

Code:

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
df = pd.read_csv('wine.data.csv')
df.head(10)
df.iloc[:,1:].describe()
for c in df.columns[1:]:
df.boxplot(c,by='Class',figsize=(7,4),fontsize=14)
plt.title("{}\n".format(c),fontsize=16)
plt.xlabel("Wine Class", fontsize=16)
plt.figure(figsize=(10,6))
plt.scatter(df['OD280/OD315 of diluted
wines'],df['Flavanoids'],c=df['Class'],edgecolors='k',alpha=0.75,s=150)
plt.grid(True)
plt.title("Scatter plot of two features showing the \ncorrelation and class seperation",fontsize=15)
plt.xlabel("OD280/OD315 of diluted wines",fontsize=15)
plt.ylabel("Flavanoids",fontsize=15)
plt.show()
def correlation_matrix(df):
from matplotlib import pyplot as plt
from matplotlib import cm as cm

fig = plt.figure(figsize=(16,12))
ax1 = fig.add_subplot(111)
cmap = cm.get_cmap('jet', 30)
cax = ax1.imshow(df.corr(), interpolation="nearest", cmap=cmap)

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 53 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

ax1.grid(True)
plt.title('Wine data set features correlation\n',fontsize=15)
labels=df.columns
ax1.set_xticklabels(labels,fontsize=9)
ax1.set_yticklabels(labels,fontsize=9)
# Add colorbar, make sure to specify tick locations to match desired ticklabels
fig.colorbar(cax, ticks=[0.1*i for i in range(-11,11)])
plt.show()

correlation_matrix(df)
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
X = df.drop('Class',axis=1)
y = df['Class']
X = scaler.fit_transform(X)
dfx = pd.DataFrame(data=X,columns=df.columns[1:])
dfx.head(10)
dfx.describe()
from sklearn.decomposition import PCA
pca = PCA(n_components=None)
dfx_pca = pca.fit(dfx)
plt.figure(figsize=(10,6))
plt.scatter(x=[i+1 for i in range(len(dfx_pca.explained_variance_ratio_))],
y=dfx_pca.explained_variance_ratio_,
s=200, alpha=0.75,c='orange',edgecolor='k')
plt.grid(True)
plt.title("Explained variance ratio of the \nfitted principal component vector\n",fontsize=25)
plt.xlabel("Principal components",fontsize=15)
plt.xticks([i+1 for i in range(len(dfx_pca.explained_variance_ratio_))],fontsize=15)
plt.yticks(fontsize=15)
plt.ylabel("Explained variance ratio",fontsize=15)
plt.show()
dfx_trans = pca.transform(dfx)
dfx_trans = pd.DataFrame(data=dfx_trans)
dfx_trans.head(10)
plt.figure(figsize=(10,6))

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 54 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

plt.scatter(dfx_trans[0],dfx_trans[1],c=df['Class'],edgecolors='k',alpha=0.75,s=150)
plt.grid(True)
plt.title("Class separation using first two principal components\n",fontsize=20)
plt.xlabel("Principal component-1",fontsize=15)
plt.ylabel("Principal component-2",fontsize=15)
plt.show()

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 55 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 56 / 57
 lOMoARcPSD|45333583

www.BrainKart.com

VIVA Questions

1. What is machine learning?

2. Define supervised learning
3. Define unsupervised learning
4. Define semi supervised learning
5. Define reinforcement learning
6. What do you mean by hypotheses?
7. What is classification?
8. What is clustering?
9. Define precision, accuracy and recall
10. Define entropy
11. Define regression
12. How Knn is different from k-means clustering
13. What is concept learning?
14. Define specific boundary and general boundary
15. Define target function
16. Define decision tree
17. What is ANN
18. Explain gradient descent approximation
19. State Bayes theorem
20. Define Bayesian belief networks
21. Differentiate hard and soft clustering
22. Define variance
23. What is inductive machine learning?
24. Why K nearest neighbor algorithm is lazy learning algorithm
25. Why naïve Bayes is naïve
26. Mention classification algorithms
27. Define pruning
28. Differentiate Clustering and classification
29. Mention clustering algorithms
30. Define Bias
31. What is learning rate? Why it is need.

https://play.google.com/store/apps/details?id=info.therithal.brainkart.annauniversitynotes Page: 57 / 57
 Click on Subject/Paper under Semester to enter.
Professional English Discrete Mathematics Environmental Sciences
Professional English - - II - HS3252 - MA3354 and Sustainability -
I - HS3152 GE3451
Digital Principles and
Statistics and Probability and
Computer Organization
Matrices and Calculus Numerical Methods - Statistics - MA3391
- CS3351
- MA3151 MA3251
3rd Semester
1st Semester

4th Semester
2nd Semester

Database Design and Operating Systems -

Engineering Physics - Engineering Graphics
Management - AD3391 AL3452
PH3151 - GE3251

Physics for Design and Analysis of Machine Learning -

Engineering Chemistry Information Science Algorithms - AD3351 AL3451
- CY3151 - PH3256
Data Exploration and Fundamentals of Data
Basic Electrical and
Visualization - AD3301 Science and Analytics
Problem Solving and Electronics Engineering -
BE3251 - AD3491
Python Programming -
GE3151 Artificial Intelligence
Data Structures Computer Networks
- AL3391
Design - AD3251 - CS3591

Deep Learning -
AD3501

Embedded Systems
Data and Information Human Values and
and IoT - CS3691
5th Semester

Security - CW3551 Ethics - GE3791

6th Semester

7th Semester

8th Semester

Open Elective-1
Distributed Computing Open Elective 2
- CS3551 Project Work /
Elective-3
Open Elective 3 Intership
Big Data Analytics - Elective-4
CCS334 Open Elective 4
Elective-5
Elective 1 Management Elective
Elective-6
Elective 2
 All Computer Engg Subjects - [ B.E., M.E., ] (Click on Subjects to enter)
Programming in C Computer Networks Operating Systems
Programming and Data Programming and Data Problem Solving and Python
Structures I Structure II Programming
Database Management Systems Computer Architecture Analog and Digital
Communication
Design and Analysis of Microprocessors and Object Oriented Analysis
Algorithms Microcontrollers and Design
Software Engineering Discrete Mathematics Internet Programming
Theory of Computation Computer Graphics Distributed Systems
Mobile Computing Compiler Design Digital Signal Processing
Artificial Intelligence Software Testing Grid and Cloud Computing
Data Ware Housing and Data Cryptography and Resource Management
Mining Network Security Techniques
Service Oriented Architecture Embedded and Real Time Multi - Core Architectures
Systems and Programming
Probability and Queueing Theory Physics for Information Transforms and Partial
Science Differential Equations
Technical English Engineering Physics Engineering Chemistry
Engineering Graphics Total Quality Professional Ethics in
Management Engineering
Basic Electrical and Electronics Problem Solving and Environmental Science and
and Measurement Engineering Python Programming Engineering