Decision tree

The document provides an overview of decision trees, including their structure, advantages, disadvantages, and applications in various fields such as banking and education. It also discusses the implementation of decision trees using Python, focusing on key concepts like Gini index, entropy, and information gain. Additionally, it outlines the steps for calculating entropy and information gain to determine the best features for decision-making.

Uploaded by

chodanker15

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PPTX, PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

0 views

Decision tree

Uploaded by

chodanker15

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PPTX, PDF, TXT or read online on Scribd

You are on page 1/ 56

GOA COLLEGE OF ENGINEERING

Affiliated to Goa University

INFORMATION TECHNOLOGY DEPARTMENT

UNIT 2
Vision
Impart high quality knowledge and skills to students in the field of Information Technology ,motivate research, encourage industry consultancy projects and
nurture human values and life skills.

Machine
learning
algorithms (2)
Analysing, Visualizing and Applying Data Science Ms. Seeya Gude
Semester VIII Asst. Professor
B.E.(I.T.) Information Technology Department
Academic Year 2024-25 Goa College of Engineering
MINOR DEGREE DATA SCEINCE SEM 8
06/07/2025 1
(By Ms. Seeya Gude)
Decision trees
A decision tree is a graphical representation of different options for
solving a problem and show how different factors are related. It has a
hierarchical tree structure starts with one main question at the top
called a node which further branches out into different possible
outcomes where:
• Root Node is the starting point that represents the entire dataset.
• Branches: These are the lines that connect nodes. It shows the flow from one
decision to another.
• Internal Nodes are Points where decisions are made based on the input
features.
• Leaf Nodes: These are the terminal nodes at the end of branches that
represent final outcomes or predictions

06/07/2025 MINOR DEGREE DATA SCEINCE SEM 8 (By Ms. Seey 2

a Gude)
Example of Decision tree

06/07/2025 MINOR DEGREE DATA SCEINCE SEM 8 (By Ms. Seey 3

a Gude)
Decision trees
Advantages of Decision Trees
• Simplicity and Interpretability: Decision trees are straightforward and
easy to understand. You can visualize them like a flowchart which
makes it simple to see how decisions are made.
• Versatility: It means they can be used for different types of tasks can
work well for both classification and regression
• No Need for Feature Scaling: They don’t require you to normalize or
scale your data.
• Handles Non-linear Relationships: It is capable of capturing non-
linear relationships between features and target variables.
06/07/2025 MINOR DEGREE DATA SCEINCE SEM 8 (By Ms. Seey 4
a Gude)
Decision trees
Disadvantages of Decision Trees
• Overfitting: Overfitting occurs when a decision tree captures noise
and details in the training data and it perform poorly on new data.
• Instability: instability means that the model can be unreliable slight
variations in input can lead to significant differences in predictions.
• Bias towards Features with More Levels: Decision trees can become
biased towards features with many categories focusing too much on
them during decision-making. This can cause the model to miss out
other important features led to less accurate predictions .

06/07/2025 MINOR DEGREE DATA SCEINCE SEM 8 (By Ms. Seey 5

a Gude)
Decision trees
• Applications of Decision Trees
• Loan Approval in Banking:
• Medical Diagnosis:
• Predicting Exam Results in Education

06/07/2025 MINOR DEGREE DATA SCEINCE SEM 8 (By Ms. Seey 6

a Gude)
Decision trees using Python
Key Components of Decision Trees in Python
• Root Node: The decision tree’s starting node, which stands for the complete
dataset.
• Branch Nodes: Internal nodes that represent decision points, where the data
is split based on a specific attribute.
• Leaf Nodes: Final categorization or prediction-representing terminal nodes.
• Decision Rules: Rules that govern the splitting of data at each branch node.
• Attribute Selection: The process of choosing the most informative attribute
for each split.
• Splitting Criteria: Metrics like information gain, entropy, or the Gini Index are
used to calculate the optimal split.
06/07/2025 MINOR DEGREE DATA SCEINCE SEM 8 (By Ms. Seey 7
a Gude)
Decision trees using Python
Pseudocode of Decision tree
1.Find the best attribute and place it on the root node of the tree.
2.Now, split the training set of the dataset into subsets. While making the
subset make sure that each subset of training dataset should have the same
value for an attribute.
3.Find leaf nodes in all branches by repeating 1 and 2 on each subset.

06/07/2025 MINOR DEGREE DATA SCEINCE SEM 8 (By Ms. Seey 8

a Gude)
Decision trees using Python

• Key concept in Decision Tree

1. Gini index
2. Entropy
3. Information Gain

06/07/2025 MINOR DEGREE DATA SCEINCE SEM 8 (By Ms. Seey 9

a Gude)
Decision trees using Python
• Key concept in Decision Tree
• Gini index and information gain both of these methods are used
to select from the n attributes of the dataset which attribute
would be placed at the root node or the internal node.
• Gini index
• Gini Index =1−∑jj2 Gini Index =1−∑jj2
• Gini Index is a metric to measure how often a randomly chosen
element would be incorrectly identified.
• It means an attribute with lower gini index should be preferred.
• Sklearn supports “gini” criteria for Gini Index and by default, it
takes “gini” value.

06/07/2025 MINOR DEGREE DATA SCEINCE SEM 8 (By Ms. Seey 10

a Gude)
Decision trees using Python
• Key concept in Decision Tree
• Information Gain
• Definition: Suppose S is a set of instances, A is an attribute, SvSvis
the subset of s with A = v and Values(A) is the set of all possible of
A, then
• The entropy typically changes when we use a node in a Python
decision tree to partition the training instances into smaller
subsets. Information gain is a measure of this change in entropy.
• Sklearn supports “entropy” criteria for Information Gain and if we
want to use Information Gain method in sklearn then we have to
mention it explicitly.
06/07/2025 MINOR DEGREE DATA SCEINCE SEM 8 (By Ms. Seey 11
a Gude)
Decision trees using Python
• Key concept in Decision Tree
• Entropy
• If a random variable x can take N different value, the i’value xixi
with probability piipiiwe can associate the following entropy with
x:
• H(x)=−∑i=1Np(xi)log2p(xi)H(x)=−∑i=1Np(xi)log2p(xi)
• Entropy is the measure of uncertainty of a random variable, it
characterizes the impurity of an arbitrary collection of examples.
The higher the entropy the more the information content.