UNIT-1:

1. Machine learning goal and applications:

Machine learning is a subset of artificial intelligence (AI) that focuses on
developing algorithms and statistical models that enable computers to learn
from and make predictions or decisions based on data, without being explicitly
programmed to perform specific tasks. In other words, it involves creating
systems that can learn from data to improve their performance over time.

The goals of machine learning can vary depending on the specific application, but
some common objectives include:

1. Prediction: Predicting future outcomes or trends based on past data. This could
include forecasting stock prices, predicting customer behaviour, or estimating
disease risk.
2. Classification: Categorizing data into different classes or groups based on its
features. For example, classifying emails as spam or non-spam, or identifying
whether a tumour is malignant or benign based on medical imaging data.
3. Clustering: Organizing data into natural groupings or clusters based on
similarities between data points. This could be used for customer segmentation
in marketing or for grouping similar documents in text analysis.
4. Pattern Recognition: Identifying patterns and relationships within data that
can be used to make decisions or extract insights. This could involve
recognizing handwriting, detecting anomalies in network traffic, or identifying
fraudulent transactions.
5. Optimization: Finding the best possible solution to a problem by iteratively
improving performance based on feedback. This is often used in areas like
recommendation systems, where the goal is to recommend items that are most
relevant to a user's preferences.

Machine learning has a wide range of applications across various industries, including:

1. Healthcare: Predicting disease outbreaks, diagnosing medical conditions,

personalizing treatment plans, and analysing medical images.
2. Finance: Fraud detection, credit scoring, algorithmic trading, and risk
management.
3. E-commerce: Recommender systems, personalized marketing, demand
forecasting, and supply chain optimization.
4. Transportation: Autonomous vehicles, route optimization, traffic prediction,
and vehicle maintenance.
5. Natural Language Processing (NLP): Language translation, sentiment
analysis, chatbots, and text summarization.
6. Image and Video Analysis: Object recognition, facial recognition, image
classification, and video surveillance.

Here are six simple issues commonly encountered in machine learning:

1. Overfitting:
o Definition: The model learns the training data too well, capturing noise
and details that do not generalize to new data.
2. Underfitting:
 o Definition: The model is too simple to capture the underlying patterns in
the data, leading to poor performance on both training and test data.
o
3. Data Quality:
o Definition: Poor quality data, such as missing values, noise, and outliers,
can significantly affect model performance.
4. Data Quantity:
o Definition: Insufficient data can lead to models that do not generalize
well to new, unseen data.
5. Bias and Variance Trade-off:
o Definition: Balancing bias (error due to overly simplistic models) and
variance (error due to models being too complex) is crucial for model
performance.
6. Interpretability:
o Definition: Complex models (like deep neural networks) can be difficult
to interpret, making it hard to understand how decisions are made.

Steps help you choose the best machine learning algorithm for your problem:

1. Identify the Problem: Determine if it’s classification, regression, or clustering.

2. Understand Your Data: Check data size, quality, and type (numerical or
categorical).
3. Match Algorithm to Data: Choose algorithms that fit the data's size and type.
4. Evaluate Performance: Split data into training/testing sets and use
appropriate metrics.
5. Consider Practical Factors: Ensure the model is understandable, fast, and
scalable.
6. Try and Improve: Test multiple algorithms, optimize settings, and compare
results.

2. Supervised learning and unsupervised learning:

3. Supervised Learning:
a. In supervised learning, the algorithm learns from labeled data, where
each training example is associated with a corresponding target label.
b. The goal is to learn a mapping from input features to output labels, based
on the relationship between the inputs and the targets in the training
data.
c. Supervised learning tasks include classification and regression.
d. Examples of supervised learning algorithms include linear regression,
logistic regression, decision trees, support vector machines (SVM), and
neural networks.
e. Example: Spam email detection. The algorithm is trained on a dataset of
emails, where each email is labeled as either spam or not spam. The
algorithm learns to distinguish between spam and non-spam emails based
on features extracted from the email content, such as keywords or
patterns.
4. Unsupervised Learning:
a. In unsupervised learning, the algorithm learns from unlabelled data,
where there are no predefined target labels.
b. The goal is to find hidden patterns or structures within the data, such as
clustering similar data points together or dimensionality reduction.
c. Unsupervised learning tasks include clustering, dimensionality reduction,
and anomaly detection.
 d. Examples of unsupervised learning algorithms include k-means clustering,
hierarchical clustering, principal component analysis (PCA), and
autoencoders.
e. Example: Customer segmentation. In this scenario, the algorithm analyses
a dataset containing customer purchase histories but without explicit
labels. It automatically identifies groups of customers with similar
purchasing behaviour, allowing businesses to target their marketing
strategies more effectively.
3.Overfitting and Underfitting:
1. Overfitting:
 Overfitting happens when a model learns the training data too well,
capturing noise or random fluctuations in the data rather than the
underlying patterns.
 This results in a model that performs well on the training data but
generalizes poorly to new, unseen data.
 Overfitting often occurs when the model is too complex relative to the
amount of training data available.
 Signs of overfitting include high accuracy on the training data but
significantly lower accuracy on the validation or test data.
 Example: Consider a scenario where you're trying to fit a polynomial
regression model to predict housing prices based on features like square
footage and number of bedrooms. If you use a high-degree polynomial
(e.g., degree 10) to fit the data, the model may capture the noise in the
training data, resulting in a curve that excessively wiggles to fit individual
data points. This model might perform well on the training set but would
likely perform poorly on new data because it's too sensitive to fluctuations
in the training data.
2. Underfitting:
 Underfitting occurs when a model is too simple to capture the underlying
structure of the data.
 The model fails to learn from the training data effectively and performs
poorly both on the training data and on new, unseen data.
 Underfitting can happen when the model is too simple or when important
features of the data are not captured by the model.
 Signs of underfitting include low accuracy on both the training and
validation/test data.
 Example: Continuing with the housing price prediction example, if you fit a
linear regression model to the data but the true relationship between the
features and the target variable is nonlinear, the model may underfit the
data. In this case, the model would be too simplistic to capture the true
relationship between the features and the housing prices, resulting in poor
performance on both the training and test datasets.

4.Discuss four types of learning:

1. Supervised learning
2. Unsupervised learning
3. Semi-Supervised Learning:
a. Semi-supervised learning lies between supervised and unsupervised
learning. It involves training a model on a dataset that contains both
labeled and unlabelled data.
b. The model learns from the labeled data while also leveraging the
unlabelled data to improve performance.
 c. Semi-supervised learning is useful when labeled data is scarce or
expensive to obtain.
d. Examples: Training a model to classify images using a combination of
labeled and unlabelled data, where only a small portion of the images
are labeled.
4. Reinforcement Learning:
a. Reinforcement learning involves training an agent to interact with an
environment and learn to make decisions in order to maximize some
notion of cumulative reward.
b. The agent learns through trial and error, receiving feedback from the
environment in the form of rewards or penalties.
c. The goal is to learn a policy that maps states of the environment to
actions that maximize the cumulative reward over time.
d. Reinforcement learning is used in various applications, including game
playing, robotics, and autonomous vehicle control.
e. Examples: Training an AI agent to play chess or go, teaching a robot to
navigate through a maze, or optimizing resource allocation in a
manufacturing process.
5. Generative and discriminative model:
1. Generative Models:
 Modeling Approach: Generative models aim to learn the joint probability
distribution of the input features and the class labels.
 Learning Process: During training, generative models estimate the
probability of observing each feature given each class, as well as the prior
probability of each class.
 Example: Naive Bayes classifier is a classic example of a generative
model. In email classification, Naive Bayes learns the probability
distribution of word frequencies in spam and non-spam emails. It models
how likely it is to see specific words in each type of email, as well as the
overall likelihood of an email being spam or non-spam.
 Usage: Generative models can be used for both classification and
generation of new data samples.
2. Discriminative Models:
 Modeling Approach: Discriminative models directly learn the decision
boundary between classes without explicitly modeling the underlying
probability distribution of the features.
 Learning Process: Discriminative models learn the conditional
probability of each class given the input features directly from the training
data.
 Example: Logistic Regression and Support Vector Machines (SVM) are
examples of discriminative models. In email classification, logistic
regression learns a linear decision boundary that separates spam emails
from non-spam emails based on the input features (e.g., word
frequencies).
 Usage: Discriminative models are primarily used for classification tasks
and don't have the capability to generate new data samples.

Comparison:

 Generative Models:
 Learns joint probability distribution.
 Can be used for classification and generation.
 Examples: Naive Bayes, Gaussian Mixture Models (GMM).
  Discriminative Models:
 Learns conditional probability directly.
 Used solely for classification.
 Examples: Logistic Regression, Support Vector Machines (SVM).
6. Decision Tree Algo:

Decision Tree Algorithm:

1. Training Phase:
 The algorithm recursively splits the dataset into subsets based on the
feature that best separates the data.
 It selects the feature that maximizes the information gain (or minimizes
impurity) at each split.
 This process continues until a stopping criterion is met, such as reaching a
maximum depth, minimum number of samples per leaf, or no further
improvement in impurity reduction.
2. Tree Construction:
 At each node, the algorithm selects the feature that best splits the data,
resulting in the highest purity (e.g., Gini impurity or entropy).
 This process is repeated recursively for each subset until all instances at a
node belong to the same class or the stopping criterion is met.
3. Prediction Phase:
 To make predictions for new instances, the algorithm traverses the tree
from the root node to a leaf node based on the feature values of the
instance.
 The class label associated with the leaf node reached by the instance is
assigned as the predicted class.

Example:
Consider a dataset containing information about customers' purchasing behavior,
including features such as age, income, and whether they purchased a product ("Yes"
or "No"). We want to predict whether a new customer will purchase the product based
on their age and income.

 The Decision Tree algorithm may split the dataset based on the feature that
provides the most information gain, such as age.
 It continues splitting the dataset into subsets based on different features until it
reaches a leaf node where all instances belong to the same class (e.g., all
customers above a certain age range purchased the product).

Issues in Decision Tree:

1. Overfitting:
 Decision trees are prone to overfitting, especially when the tree is deep or
when the dataset is noisy.
 Deep trees can memorize the training data, leading to poor generalization
on unseen data.
2. High Variance:
 Decision trees have high variance, meaning small changes in the data can
result in different tree structures.
 This can lead to instability in the model's predictions.
3. Bias towards Features with Many Levels:
  Decision trees tend to favor features with more levels or categories, as
they can create more splits and potentially overfit the data.
4. Difficulty Handling Non-Numerical Data:
 Decision trees struggle with non-numerical (categorical) data, as they
require special handling (e.g., one-hot encoding) to incorporate
categorical features.
5. Unbalanced Classes:
 Decision trees can produce biased trees when dealing with imbalanced
class distributions, where one class dominates the other.

7- Differentiation between linear and non-linear discriminative classification:

1. Linear Discriminative Classification:

 Decision Boundary: Uses a straight line (or hyperplane) to separate different

classes.
 Examples: Logistic Regression, Linear Support Vector Machine (SVM).
 Suitability: Works well when data is linearly separable (classes can be
separated by a straight line).
 Computational Complexity: Generally lower and faster to compute.
 Equation Form: y=wx+b

2. Non-Linear Discriminative Classification:

 Decision Boundary: Uses curves or more complex shapes to separate classes.

 Examples: Kernel SVM, Neural Networks.
 Suitability: Works well with complex data where classes are not linearly
separable.
 Computational Complexity: Higher due to more complex computations.
 Equation Form: Can involve higher-degree polynomials or functions

8- K nearest neighbour:

K Nearest Neighbours (KNN) is a simple yet effective supervised machine learning

algorithm used for classification and regression tasks. It is a non-parametric method
that makes predictions based on the similarity of a new data point to its neighbouring
data points.

Here's how it works:

1. Training Phase: KNN stores all available data points and their corresponding
labels.
2. Prediction Phase:
o Given a new, unseen data point, the algorithm calculates the distance
between this point and all other points in the training dataset. Common
distance metrics include Euclidean distance, Manhattan distance, or
Minkowski distance.
o It then identifies the K nearest data points (neighbors) to the new point
based on the calculated distances.
o For classification tasks, KNN takes a majority vote among the labels of the
K nearest neighbors and assigns the most frequent class label to the new
data point.
o For regression tasks, KNN takes the average of the labels of the K nearest
neighbors and assigns it as the predicted value for the new data point.
 3. Choosing K: The choice of K (the number of neighbors) is a hyperparameter
that needs to be specified before applying the algorithm. It can significantly
affect the performance of the model. A small K value may lead to noise
sensitivity and overfitting, while a large K value may lead to oversmoothing and
underfitting.

UNIT-2
1. Bagging and Boosting:

Bagging (Bootstrap Aggregating):

  Definition: Bagging creates multiple copies of the same model, each trained on
a different subset of the training data. This subset is obtained through random
sampling with replacement, meaning some data points may be repeated.
 Purpose: The goal of bagging is to reduce overfitting and variance in
predictions. By training models on diverse subsets of data and averaging their
predictions, bagging creates a more robust and stable model.
 Example: Imagine you're trying to make an important decision, and you ask for
opinions from several friends. Each friend has their own perspective (like a
different subset of data), and by considering all opinions, you get a more
balanced decision that's less likely to be influenced by individual biases.
 Popular Algorithm: Random Forest is a well-known bagging algorithm that
constructs multiple decision trees, each trained on a random subset of the
training data. The final prediction is made by averaging the predictions of all
trees.

Boosting:

 Definition: Boosting trains a series of models sequentially, with each model

focusing more on instances that were misclassified by previous models. It
assigns higher weights to misclassified instances and lower weights to correctly
classified instances.
 Purpose: Boosting aims to improve model performance by iteratively
correcting mistakes made by previous models. It's like learning from your
mistakes and getting better at a task with each attempt.
 Example: Think of it as climbing a hill with small steps. At each step, you
correct the mistakes made in the previous steps, gradually bringing you closer
to the top (better model performance).
 Popular Algorithm: AdaBoost (Adaptive Boosting) is a widely used boosting
algorithm. In AdaBoost, each weak learner (e.g., a decision stump) focuses on
instances that were misclassified by previous learners, allowing the model to
learn from its mistakes and improve over iterations.

2. AdaBoost Algo:

AdaBoost (Adaptive Boosting):

 Idea: AdaBoost is a boosting algorithm that combines multiple weak learners

(e.g., simple decision trees) to create a strong learner. It focuses on instances
that are hard to classify in each iteration.
 Process:
 Initially, each instance in the training data is assigned an equal weight.
 It trains a weak learner (e.g., a decision stump) on the data, giving more
importance to instances that were misclassified in the previous iterations.
 It repeats this process for a set number of iterations, adjusting the
instance weights at each step to focus more on difficult instances.
 Each weak learner contributes to the final prediction based on its
performance, with more accurate learners having a higher influence.
 Final Prediction:
 AdaBoost combines the predictions of all weak learners to make the final
prediction. It weighs each weak learner's prediction based on its
performance, giving more weight to those with better accuracy.

 Example:
  Suppose we have a dataset of emails labeled as spam or not spam.
AdaBoost trains multiple weak learners, such as decision stumps, on the
data. It iteratively adjusts the instance weights, giving more importance to
misclassified emails in each iteration. Finally, it combines the predictions
of all weak learners to classify new emails as spam or not spam.

3. SVM:

Support Vector Machine (SVM):

 Purpose: SVM is a powerful algorithm used for classification tasks in machine

learning.
 How it works: SVM finds the best line or boundary (hyperplane) to separate
different classes in the data. This boundary is chosen to maximize the margin
(distance) between the classes, allowing for better generalization to unseen
data.
 Key idea: The key idea behind SVM is to find the hyperplane that maximizes
the margin between the closest data points of different classes, known as
support vectors. By maximizing the margin, SVM aims to find the most robust
decision boundary.
 Example: Imagine a dataset of flowers with different features like petal length
and width. SVM can help classify these flowers into different species, such as
"Iris-setosa" and "Iris-versicolor", by finding the best line or boundary that
separates them in the feature space.
 Application: SVM finds applications in various fields, including text
classification (e.g., spam detection), image recognition (e.g., object detection),
and financial forecasting (e.g., stock price prediction). It's widely used due to its
effectiveness in handling high-dimensional data and its ability to generalize well
to unseen examples.

4. Naïve Bayes Theorem:

How Naive Bayes is useful in machine learning:

1. Fast and Efficient:

o Quick to train and make predictions, suitable for large datasets.
2. High Dimensional Data:
o Handles a large number of features well due to the independence
assumption.
3. Works with Categorical Data:
o Effective for problems involving categorical data, like text classification.
4. Multi-Class Classification:
o Easily manages classification tasks involving more than two classes.
5. Simple Implementation:
o Easy to implement and understand, making it a good starting point for
beginners.
6. Real-World Applications:
o Widely used for spam detection, sentiment analysis, and document
classification.
Limitations of perceptron:

1. Linear Separation Only: Perceptrons can only handle problems where the
data can be separated by a straight line or plane. If the data is more
complicated, perceptrons won't work well.
2. Binary Output: Perceptrons give only yes or no answers. They can't tell you
probabilities or give you a range of outputs like other models can.
3. Shallow Structure: Perceptrons are like a single layer of neurons. They can't
understand complex patterns that need many layers to figure out.
4. Sensitive to Scaling: If your data has different scales (like weight and height),
perceptrons might get confused. They need all the input data to be in a similar
range.
5. Limited Complexity: Because they're simple, perceptrons can't handle
complicated tasks. They struggle with things like recognizing faces or
understanding language.
6. Noise Sensitivity: If your data has mistakes or outliers, perceptrons might
learn the wrong thing. They're not good at handling messy data.