Bayes Theorem
Bayes Theorem
Bayes Theorem
Bayes’ theorem (also known as Bayes’ rule) is a useful tool for calculating conditional
probabilities. It relates the probability of the occurrence of an event to the occurrence or non-
occurrence of an associated event.
Let A1, A2, ---, An be a set of mutually exclusive events that together form the sample space S.
Let B be any event from the sample space, such that P (B) > 0. Then,
!(#k∩ ')
P (Ak| B) =
)(#1 ∩ ' )+ !( #2 ∩ ' )+ --- +!(#. ∩ ')
You should consider Bayes’ theorem when the following conditions exist.
§ The sample space is partitioned into a set of mutually exclusive events {A1, A2,. . . An}.
§ Within the sample space, there exists an event B, for which P (B) > 0.
§ The analytical goal is to compute a conditional probability of the form: P (Ak | B).
§ You know at least one of the two sets of probabilities described below.
• P( Ak ∩ B ) for each Ak
• P( Ak ) and P( B | Ak ) for each Ak
Example 1- An urn B1 contains 2 white and 3 black balls and another urn B2 contains 3 white
and 4 black balls. One urn is selected at random and a ball is drawn from it. If the ball drawn is
found black, find the probability that the urn chosen was B1.
Solution:
3
Then P (E1) = P (E2) =
4
Let B denote the event that the ball chosen from the selected urn is black.
5 7
P (B | E1) = and P (B | E2) =
6 8
By Bayes’ theorem
Example 2- Marie is getting married tomorrow, at an outdoor ceremony in the desert. In recent
years, it has rained only 5 days each year. Unfortunately, the weatherman has predicted rain for
tomorrow. When it actually rains, the weatherman correctly forecasts rain 90% of the time.
When it doesn't rain, he incorrectly forecasts rain 10% of the time. What is the probability that it
will rain on the day of Marie's wedding?
Solution:
The sample space is defined by two mutually-exclusive events - it rains or it does not rain.
Additionally, a third event occurs when the weatherman predicts rain. Notation for these events
appears below.
We want to know P (A1 | B), the probability it will rain on the day of Marie's wedding, given a
forecast for rain by the weatherman. The answer can be determined from Bayes' theorem, as
shown below.
P( A1 ) P( B | A1 )
P( A1 | B ) =
P( A1 ) P( B | A1 ) + P( A2 ) P( B | A2 )
P( A1 | B ) = (0.014)(0.9) / [ (0.014)(0.9) + (0.986)(0.1) ]
P( A1 | B ) = 0.111
a. If an ELT is randomly selected from the general population of all ELTs, find the probability
that it was made by the Altigauge Manufacturing Company.
b. If a randomly selected ELT is then tested and is found to be defective, find the probability
that it was made by the Altigauge Manufacturing Company.
Solution:
a. If an ELT is randomly selected from the general population of all ELTs, the probability that it
was made by Altigauge is 0.8 (because Altigauge manufactures 80% of them).
b. If we now have the additional information that the ELT was tested and was found to be
defective, we want to revise the probability from part (a) so that the new information can be
used. We want to find the value of P (A|D), which is the probability that the ELT was made by
the Altigauge Company given that it is defective. Based on the given information, we know these
probabilities:
Here is Bayes' theorem extended to include three events corresponding to the selection of ELTs
from the three manufacturers (A, B, C):
!(#).!(A|#)
P (A|D) =
B !(#).!CDEAGH+ B !(').!CDEBGH+ [ ! (K).!CDECG]
N.ON .N.N7
=
[N.ON .N.N7]+ [N.36 .N.NP]+ [N.N6 .N.NQ]
= 0.703 (rounded)
Example 4- In Orange County, 51% of the adults are males. (It doesn't take too much advanced
mathematics to deduce that the other 49% are females.) One adult is randomly selected for a
survey involving credit card usage.
b. It is later learned that the selected survey subject was smoking a cigar. Also, 9.5% of males
smoke cigars, whereas 1.7% of females smoke cigars (based on data from the Substance Abuse
and Mental Health Services Administration). Use this additional information to find the
probability that the selected subject is a male.
Solution:
a. Before using the information given in part b, we know only that 51% of the adults in Orange
County are males, so the probability of randomly selecting an adult and getting a male is given
by P(M) = 0.51.
P (Mʹ) = 0.49 because 49% of the adults are females (not males)
P (C|M) = 0.095 because 9.5% of the males smoke cigars (That is, the probability of getting
someone who smokes cigars, given that the person is a male, is 0.095.)
P (C| Mʹ) = 0.017 because 1.7% of the females smoke cigars (That is, the probability of
getting someone who smokes cigars, given that the person is a female, is 0.017.)
Let's now apply Bayes' theorem by using the preceding formula with M in place of A, and C in
place of B. We get the following result:
! (R).! (K|R)
P (M|C) =
B ! (R).! CCEMGH+ [ ! (RT ).! (K|RT )]
N.63 .N.NQ6
=
[N.63 .N.NQ6]+ [ N.7Q .N.N38]
= 0.85329341
= 0.853 (rounded)
Example 5- The urns contain 6 green, 4 black; 4 green, 6 black and 5 green, 5 black balls
respectively. Randomly selected an urn and a ball is drawn from it. If the ball drawn is Green,
find the probability that it is drawn from the first urn.
Solution:
Since there are three urns and one of the three urns is chosen at random, therefore
3
P (E1) = P (E2) = P (E3) =
5
If E1 is already occurred, then urn first has been chosen which contains 6 Green and 4 Black
P
balls. The probability of drawing a green ball from it is
3N
P
So, P (A|E1) =
3N
7 6
Similarly, we have P (A|E2) = and P (A|E3) =
3N 3N
we are required to find P (E|A), i.e. given that the ball drawn is Green, what is the probability
that it is drawn from the first urn.
! (91).! (# |91)
P (E | A) =
[ ) (91).) (U |91)]+[! (92).! (# |92)]+[ ) (95 ) .! (# | 93)]
: W
×
= :X
= : W : ? : >
× + ×
= :X = :X = :X
+ ×
P
=
36
4
=
6
Practice Problems:
Question 1: A speaks the truth 4 out of 5 times. A die is tossed. A reports that it is a 6. What are
the chances that there actually was a 6?
Question 2: A is known to tell the truth in 5 cases out of 6 and he states that a white ball was
drawn from a bag containing 8 blacks and 1 white ball. Find the probability that the white ball
was drawn.
Question 3: A doctor is called to see a sick child. The doctor has prior information that 90% of
sick children in that neighborhood have the flu, while the other 10% are sick with measles. A
well-known symptom of measles is a rash P (R|M) = .95. However, occasionally children with
flu also develop rash, so that P (R|F) = 0.08. Upon examining the child, the doctor finds a rash.
What is the probability that the child has measles?
Question 4: A company manufactures T.Vs at two different plants A and B. Plant ‘A’ produces
80% and ‘B’ produces 20% of the total production. 85 out of 100 T.Vs produced at plant ’A’
meet the quality standards while 65 out of 100 T.Vs produced at plant ‘B’ meet the quality
standards. A T.V produced by the company is selected at random and is not found to meeting the
quality standard. Find the probability that the selected T.V was manufactured by the plant ‘B’.
Question 5: At a certain university, 4% of men are over 6 feet tall and 1% of women are over 6
feet tall. The total student population is divided in the ratio 3:2 in favour of women. If a student
is selected at random from among all those over 6 feet tall, what is the probability that the
student is a woman?
Question 6: An engineering company advertises a job in three papers, A, B, C. It is known that
these papers attract undergraduate engineering readerships in the proportions 3:2:1. The
probabilities that an engineering graduate sees and replies to the job advertisements in these
papers are 0.002, 0.001 and 0.005 respectively. Assume that the undergraduate sees only one job
advertisement.
a. If the engineering company receives only one reply to its advertisements, calculate the
probability that the applicant has seen the job advertised in place
I- A
II- B
III- C
b. If the company receives two replies, what is the probability that both applicants saw the job
advertised in paper A?
Question 7: Machines A and B produces 10% and 90% respectively of the production of a
component intended for the motor industry. From experience, it is known that the probability that
machine A produces a defective component is 0.01 while the probability that machine B
produces a defective component is 0.05. If a component is selected at random from a day’s
production and is found to be defective, find the probability that it was made by
(a) Machine A
(b) Machine B
Answers:
7
1.
Q
6
2.
35
3.0.568 ≈ 0.57
8
4.
3Q
5
5.
33
6. a.
3
I-
5
3
II-
7
6
III-
34
3
b.
Q
7. (a) 0.02
(b) 0.978 ≈ 0.98
REFERNCES:
http://math.tutorvista.com/statistics/bayes-theorem.html
http://stattrek.com/probability/bayes-theorem.aspx
http://faculty.washington.edu/tamre/BayesTheorem.pdf
http://www.cut-the-knot.org/Probability/BayesTheorem.shtml
http://nucinkis-lab.cc.ic.ac.uk/HELM/...35/35_4_tot_prob_bayes_thm.pdf