Scribd
Upload
199 views4 pages

Assignment 3: Introduction To Machine Learning Prof. B. Ravindran

This document contains a summary of key concepts from an introduction to machine learning course, including: 1. If the decision boundary separating two classes is linear, at least one of the discriminant functions can be linear, but both do not need to be. 2. The minimum number of basis transformations required to avoid masking for K classes is K-1. 3. Linear discriminant analysis maximizes the variance between classes relative to the within-class variance. 4. Principal component analysis and logistic regression cannot achieve zero training error on non-linearly separable data, even without additional complexity.

Uploaded by

Vijayramasamy
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
199 views4 pages

Assignment 3: Introduction To Machine Learning Prof. B. Ravindran

This document contains a summary of key concepts from an introduction to machine learning course, including: 1. If the decision boundary separating two classes is linear, at least one of the discriminant functions can be linear, but both do not need to be. 2. The minimum number of basis transformations required to avoid masking for K classes is K-1. 3. Linear discriminant analysis maximizes the variance between classes relative to the within-class variance. 4. Principal component analysis and logistic regression cannot achieve zero training error on non-linearly separable data, even without additional complexity.

Uploaded by

Vijayramasamy
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

Assignment 3

Introduction to Machine Learning


Prof. B. Ravindran
1. Given that the decision boundary separating two classes is linear, what can be inferred about
the discriminant functions of the two classes?
(a) Both discriminant functions have to be necessarily linear.
(b) At least one of the discriminant functions is linear.
(c) Both discriminant functions can be non-linear.
Sol. (c)
2. We discussed the concept of masking in video lectures. What are the minimum number of
basis transformations required in order to avoid masking for K classes?
(a) K
(b) K − 1
(c) K 2
(d) K(K − 1)/2
Sol. (b)
Refer to the videos of Linear Classifier
eα0 +αx eβ0 +βx
3. Consider the function f1 (x) = 1+eα0 +αx
and f2 (x) = 1+eβ0 +βx
shown in the figure below:

Figure 1: f1 (x) Figure 2: f2 (x)

Which of the following is correct?


(a) 0 < β < α
(b) 0 < α < β
(c) α < β < 0
(d) β < α < 0
Sol. (d)

1
4. Which of the following is correct about linear discriminant analysis ?

(a) It minimizes the variance between the classes relative to the within class variance.
(b) It maximizes the within class variance relative to the variance between classes.
(c) It maximizes the variance between the classes relative to the within class variance.
(d) None of these

Sol. (c)
Refer to the videos
5. (2 marks) Consider the case where two classes follow Gaussian distribution which are cen-
tered at (−1, 2) and (1, 4) and have identity covariance matrix. Which of the following is the
separating decision boundary?

(a) y − x = 3
(b) x + y = 3
(c) x + y = 6
(d) (b) and (c) are possible
(e) None of these
(f) Can not be found from the given information
Sol. (b)
As the distribution is Gaussian and have identity covariance (which are equal), the separating
boundary will be linear. The decision boundary will be orthogonal to the line joining the
centers and will pass from the midpoint of centers.
6. Consider the following data with two classes. The color indicates different class.

Which of the following models (with NO additional complexity) can achieve zero training error
for classification?
(a) LDA
(b) PCA

2
(c) Logistic regression
(d) None of these
Sol. (d)
All the classifiers in option (a), (b) and (c) are linear. The training data is linearly non-
separable.
7. Which of the following technique is well known to utilize class labels in feature selection for
classification?
(a) LDA
(b) PCA (by dimensionality reduction)
(c) both (a) and (b)
(d) None of these
Sol. (a)
PCA does not use the class labels. It chooses the directions with maximum variance in the
data.
8. We discussed the use of MLE for the estimation of parameters of logistic regression model. We
used which of the following assumptions to derive the likelihood function ?
(a) independence among the class labels
(b) independence among each training sample
(c) independence among the parameters of the model
(d) None of these
Sol. (b)
9. Consider the following distribution of training data:

Which method would you choose for dimensionality reduction?

3
(a) Linear Discriminant Analysis
(b) Principal Component Analysis
(c) (a) and (b) perform very poorly, so have to choose Quadratic Discriminant Analysis
(d) (a) or (b) are equally good
(e) None of these

Sol. (d)
The direction of maximum variance is along the direction X1 = X2 . The projected points in
this direction can be easily classified into two classes correctly.
LDA can find a linearly separable decision boundary as the data is linearly spearable.

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy