CHAPTER 5

LEARNING AGENTS

Learning is the process by which one

entity acquires knowledge.
 Chapter’s point of discussion

Introduction

Learning from Examples/Observation

Knowledge in Learning

Learning Probabilistic Models

Neural Networks

2
 Introduction
• When all intelligence in an agent has been built-in by the agent's
designer, the agent does the best given the way it was
programmed to act.
– But this is not necessarily the best approach for the agent with
incomplete knowledge of its environment.
• Whenever the designer has incomplete knowledge of the
environment that the agent will live in, learning is the only way
that the agent can acquire what it needs to know.
– Learning provides autonomy.
– Learning provides a good way to build high performance
systems by giving a learning system experience in the
application domain.
3
 Introduction(cont’d…)
• The idea behind learning is that percepts should be used not only
for acting, but also for improving the agent's ability to act in the
future.

• Learning takes place as a result of the interaction between the

agent and the world, and from observation by the agent of its own
decision-making processes.
– Learning can range from trivial memorization of experience to
the creation of entire scientific theories.

4
 Learning from Examples/Observation
• Learning from example is the process of acquiring generalized
knowledge from examples or instances of some classes.
• As human learn from past experiences, a computer does not have
“experiences”.
– A computer system learns from data, which represent some
“past experiences” of an application domain.
– Learning agent is able to perform tasks, analyze performance,
and look for new ways to improve its performance by itself by
considering its past experiences.
• Learning agent starts with some basic knowledge and then able to
act and adapt autonomously, through learning, to improve its own
performance.
5
 Learning from Examples/Observation(cont’d…)
• Machine learning: is the subfield of AI concerned with programs
that automatically learn from experience without being explicitly
programmed.
– Learning is the execution of a computer program to optimize
the parameters of a model using the training data or past
experience.
– The model may be:
• Predictive: to make predictions in the future, or
• Descriptive: to gain knowledge from data, or both.
• Machine learning uses the theory of statistics in building
mathematical models, because the core task is making inference
from a sample.
6
 Learning from Examples/Observation(cont’d…)
• Based on data types machine learning could be: Supervised or
Unsupervised.
• Supervised Machine learning:
– learns from examples.
– Supervision: the data (observations, measurements, etc.) are
labeled with pre-defined classes.
– Test data are classified into these classes too.
• Unsupervised Machine learning
– The data have no target attribute.
– We want to explore the data to find some intrinsic structures in
them.
7
 Learning from Examples/Observation(cont’d…)
• Data:
– a set of data records: also called examples, instances or cases.
It is described by k attributes: A1, A2, … Ak and
– a class: each example is labelled with a pre-defined class.
• Goal: to learn a classification model from the data that can be used
to predict the classes of new (future or test) cases/instances.
• Example: A bank receives thousands of applications for loan. Each
application contains information about an applicant, age, Marital
status, annual salary, outstanding debts, credit rating, etc.
• Problem: to decide whether an application should approved, or to
classify applications into two categories, approved(Yes) and not
approved(No).
8
Learning from Examples/Observation(cont’d…)

9
 Learning from Examples/Observation(cont’d…)
1. Learn a classification model from the data
2. Use the model to classify future loan applications into
– Yes (approved) and
– No (not approved)
• What is the class for following case/instance?

10
 Learning from Examples/Observation(cont’d…)
• Supervised machine learning has two steps:
– Learning (training): Learn a model using the training data
– Testing: Test the model using unseen test data to assess the
model accuracy
Number of correct classifications
Accuracy 
Total number of test cases

11
 Learning from Examples/Observation(cont’d…)
• Given a data set D, a task T, and a performance measure M,
– A computer system is said to learn from D to perform the task T if after
learning the system’s performance on T improves as measured by M.
– In other words, the learned model helps the system to perform T better as
compared to no learning.
• Example
– Data: Loan application data
– Task: Predict whether a loan should be approved or not.
– Performance measure: accuracy.
 No learning: classify all future applications (test data) to the majority class
(i.e., Yes): Accuracy = 9/15 = 60%.
 We can do better than 60% with learning.

12
 Learning from Examples/Observation(cont’d…)
• Fundamental assumption of learning from examples:
– The distribution of training examples is identical to the
distribution of test examples (including future unseen
examples).
• In practice, this assumption is often violated to certain
degree.
• Strong violations will clearly result in poor classification
accuracy.
• To achieve good accuracy on the test data, training
examples must be sufficiently representative of the test
data.
13
 Example
• Consider the following training set

• Show all necessary steps to determine the class of test set “today =
(Sunny, Hot, Normal, F)” using Naïve Bayes algorithm

14
 Example(cont’d…)
• Naïve Bayes: classifier is a powerful classification algorithm.
– It assumes that the presence of a feature in a class is unrelated
to any other feature (i.e., each feature makes an independent
and equal contribution to the outcome).
– Rule:

– Where 𝑷 ( 𝒙|𝒚 ) ∗ 𝑷 (𝒚 )
𝑷 (𝒚∨ 𝒙)=
• y represents class and 𝑷 (𝒙 )
• x represents conditions

15
 Example(cont’d…)
• From training set, we can generate the following information.

16
 Example(cont’d…)
• Probability of play is given by:

and probability to not play is given by:

• Since, P(today) is common in both probabilities, we can

ignore P(today) and find proportional probabilities as:

17
 Example(cont’d…)
• Since,but if we add the number we have, we cannot get it.
– Thus, these numbers can be converted into a probability by
making the sum equal to 1 (normalization):

• Since , playing of game in a given condition is predicted as possible.

18
 Knowledge in Learning
• Knowledge is essential for learning agent to improve its
performance and effectiveness.
• Learning agent can acquire knowledge from various sources
including but not limited to:
– Data
• Historical Data: Past data that can provide insights and
patterns.
• Real-Time Data: Continuous streams of data that allow for
adaptive learning.
• Synthetic Data: Data generated through simulations or
algorithms when real data is scarce.
19
 Knowledge in Learning(cont’d…)
• Human Expertise
– Expert Systems: knowledge from human experts encoded into rules or
frameworks.
– Crowdsourcing: gathering knowledge and insights from a large group of
people.
• Feedback Mechanisms
– Reinforcement Signals: rewards and penalties that guide the agent’s learning
process.
– User Interaction: direct feedback from users that can refine the agent’s
understanding.
• Environment Interaction
– Observations: learning from the environment through interaction.
– Simulation: creating simulated environments to test and refine strategies
without real-world consequences.
20
 Knowledge in Learning(cont’d…)
• Knowledge Bases
– Databases: structured collections of information that the agent
can query.
– Ontologies: formal representations of knowledge within a
domain, defining concepts and relationships.
• Social Media and Online Content
– Web Scraping: collecting data from websites, forums and social
media platforms for trend analysis and sentiment detection.
– User-Generated Content: leveraging content created by users
to enhance understanding and knowledge.

21
 Learning Probabilistic Models
• Drawback of pure logic:
– Laziness: too much work to list all conditions needed to ensure
an exception less rule
– Theoretical ignorance: science has no complete theory for the
domain.
– Practical ignorance: even if we know all the rules, we may be
uncertain about them.
• We only have a degree of belief in them
– AI system based on pure logic cannot do something, that it has
not been explicitly told.

22
 Learning Probabilistic Models(cont’d…)
• In real life, it is not always possible to determine the state of the
environment as it might not be clear.
– Uncertainty is everywhere.
• Causes of uncertainty:
– Information occurred from unreliable sources.
– Experimental Errors
– Equipment fault
– Temperature variation
– Climate change.
• Agent may need to handle uncertainty and deal with.

23
 Learning Probabilistic Models(cont’d…)
• Uncertainty quantification: is the science of measurable
description and reduction of uncertainties in both computational
and real world applications.
– It tries to determine how likely certain outcomes are, if some
aspects of the system are not exactly known.
• We need quantitative notion of uncertainty to reason in presence
of uncertainty.
• We use probability theory to represent and process uncertainty.

24
 Learning Probabilistic Models(cont’d…)
• Probability is defined as a chance that an uncertain event will occur.
– It is the numerical measure of the likelihood that an event will occur.
– The value of probability always remains between 0 and 1 that
represent ideal uncertainties.
• 0 ≤ P(A) ≤ 1, where P(A) is the probability of an event A.
• P(A) = 0, indicates that an event A will never occur.
• P(A) =1, indicates that an event A will occur for sure.
• We can find the probability of an uncertain event by using the below
formula.

• P(¬A) = probability of a not happening event.

– P(¬A) + P(A) = 1.
25
 Learning Probabilistic Models(cont’d…)
• Event: is each possible outcome of a variable
• Sample space: is the collection of all possible events.
• Random variables: are used to represent the events and objects in
the real world.
• Prior probability: is probability computed before observing new
information.
• Posterior Probability: is calculated after all evidence or
information has taken into account.
– It is a combination of prior probability and new information.

26
 Learning Probabilistic Models(cont’d…)
• Conditional probability is a probability of occurring an event when
another event has already happened.
• Let's suppose, we want to calculate the event A when event B has
already occurred, "the probability of A under the conditions of B",
it can be written as:

• Where: P(A⋀B)= Joint probability of A and B

P(B)= Marginal probability of B.
• If the probability of A is given and we need to find the probability
of B, then it will be given as:

27
 Learning Probabilistic Models(cont’d…)
• Example: In a class, there are 70% of the students who like English
and 40% of the students who likes English and mathematics, and then
what is the percent of students those who like English also like
mathematics?
Solution: Let:
– A is an event that a student likes Mathematics &
– B is an event that a student likes English.

• Hence, 57% are the students who like English also like Mathematics.
28
 Using probability theory to make decision
• Example: Bob is thinking whether he should take insurance on a
shipment from Dubai to Ethiopia.
– If the ship does not encounter a storm, the shipment arrives on
time in Ethiopia, and Bob will earn 10,000 ETB.
– If the ship encounters a storm, the shipment will be delayed
and Bob will earn only 8000 ETB.
• The insurance company want Bob to pay 1000 ETB for a full
coverage. Should Bob buy the policy when storm is 20%?

29
 Using probability theory to make decision(cont’d…)
• Solution:
– Bob’s expected earnings in the absence of insurance
= (0.2)(8000) + 0.8 (10,000)
= 1600 + 8000
= 9600 ETB
– Bob’s expected earnings if he purchases insurance
= 10,000 – 1000
= 9000 ETB
– Bob prefer to perform shipment without insurance.

30
 Neural Networks
• Neural networks are artificial systems that were inspired by
biological neural networks.
• A neural network is a system that is designed to operate like a
human brain.
• A neural network is usually described as having different layers.
– Input layer: picks up the input signals and passes them to the
next layer.
– Hidden layer: does all kinds of calculations and feature
extractions.
• Often, there will be more than one hidden layer.
– Output layer: delivers the final result.
31
 Neural networks(cont’d…)
• Activation functions transform the weighted sum of inputs that
goes into the artificial neurons.
– These functions should be non-linear to encode complex
patterns of the data.
• The most popular activation functions are:
– Sigmoid
– Tanh and
– ReLU.

32
 Neural networks(cont’d…)
• Sigmoid Function: is used when the model is predicting
probability.

• Threshold Function: is used when we don’t want to worry about

the uncertainty in the middle.

33
 Neural Networks(cont’d…)
• The training process consists of the following steps:
• Forward Propagation: take the inputs, multiply by the weights (just
use random numbers as weights), pass the result through a sigmoid
formula to calculate the neuron’s output.
– The Sigmoid function is used to normalize the result between 0
and 1: 1/(1 + e-x)
• Back Propagation: calculate the error (i.e., the difference between the
actual output and the expected output.
– Depending on the error, adjust the weights by multiplying the error
with the input and again with the gradient of the Sigmoid curve:
• Weight += Error Input Output (1-Output), here Output (1-
Output) is derivative of sigmoid curve.
34
Neural Networks(cont’d…)

35
 Neural networks(cont’d…)
• Example: consider the following neural network structure with the
input X1 and X2:

• Calculate the value of Z when activation function is sigmoid.

Assume there is no bias.
• Calculate the error, if expected result is 0.80

36
 Types of Neural Networks
• Feed-forward Neural Network
• Recurrent Neural Network
• Convolution Neural Network
• Modular Neural Network
And many more

• Reading assignment:
– Read working nature of different types of neural networks

37
Thank You!

38