Probability Topics: 1.3.2 Bayes' Theorem

NPTEL- Probability and Distributions
Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 1

MODULE 1
PROBABILITY
LECTURE 5
Topics
1.3.2 Bayes Theorem

1.3.2 Bayes Theorem
The following theorem provides a method for finding the probability of occurrence of an
event in a past trial based on information on occurrences in future trials.
1.3.2 Theorem 3.4 (Bayes Theorem)
Let , , be a probability space and let
: be a countable collection of
mutually exclusive and exhaustive events with
> 0, . Then, for any event

with > 0, we have

| =
|
,
Proof. We have, for ,

| =

()

=
|
()

=
|
using Theorem of Total Probability.

Remark 3.2
(i) Suppose that the occurrence of any one of the mutually exclusive and
exhaustive events
, , causes the occurrence of an event . Given that the

event has occurred, Bayes theorem provides the conditional probability that
the event is caused by occurrence of event
, .


(ii) In Bayes theorem the probabilities
, , are referred to as prior

probabilities and the probabilities
|, , are referred to as posterior

probabilities.
To see an application of Bayes theorem let us revisit Example 3.4.
Example 3.5
Urn
1
contains 4 white and 6 black balls and urn
2
contains 6 white and 4 black balls.
A fair die is cast and urn
1
is selected if the upper face of die shows five or six dots.
Otherwise urn
2
is selected. A ball is drawn at random from the selected urn.
(i) Given that the drawn ball is white, find the conditional probability that it came
from urn
1
;
(ii) Given that the drawn ball is white, find the conditional probability that it came
from urn
2
.
Solution. Define the events:
drawn ball is white;

1
urn
1
is selected
2
urn
2
is selected
mutually exclusive & exhaustive events
(i) We have
1
| =
|
1
|
1
1
+|
2

=
4
10
2
6
4
10
2
6
+
6
10
4
6

=
1
4

(ii) Since
1
and
2
are mutually exclusive and
1

2
| = | = 1,we
have

2
| = 1
1
|
=
3
4

In the above example

1
| =
1
4
<
1
3
=
1
,


and
2
| =
3
4
>
2
3
=
2
,
i.e.,
(i) the probability of occurrence of event
1
decreases in the presence of the
information that the outcome will be an element of ;
(ii) the probability of occurrence of event
2
increases in the presence of information
that the outcome will be an element of .

These phenomena are related to the concept of association defined in the sequel.
Note that

1
| <
1

1
<
1
(),
and
2
| >
2

2
>
2
().

Definition 3.2

Let , , be a probability space and let and be two events. Events and are
said to be

(i) negatively associated if < ;
(ii) positively associated if > ;
(iii) independent if = .
Remark 3.3
(i) If = 0 then = 0 = , , i.e., if = 0 then
any event and are independent;
(ii) If > 0 then and are independent if, and only if, | = (),
i.e., if > 0, then events and are independent if, and only if, the
availability of the information that event has occurred does not alter the
probability of occurrence of event .
Now we define the concept of independence for arbitrary collection of events.



Definition 3.3

Let , , be a probability space. Let be an index set and let
: be a
collection of events in .

(i) Events
: are said to be pairwise independent if any pair of events

and
, in the collection
: are independent. i.e., if
, whenever , and ;

(ii) Let = 1, 2, , n, for some , so that
: =
1
, ,
is a
finite collection of events in . Events
1
, ,
are said to be independent if,

for any sub collection
1
, ,
of
1
, ,
= 2,3, ,

=1
=
=1
. (3.6)

(iii) Let be an arbitrary index set. Events
: are said to be
independent if any finite sub collection of events in
: forms a
collection of independent events.

Remark 3.4
(i) To verify that events
1
, ,
are independent one must verify

2
1 =
=2
conditions in (3.6). For example, to conclude that
three events
1
,
2
and
3
are independent, the following 4 = 2
3
3 1
conditions must be verified:
1

2
=
1
2
;
1

3
=
1
3
;
2

3
=
2
3
;

1

2

3
=
1
3
.

(ii) If events
1
, ,
are independent then, for any permutation

1
, ,
of
1, , , the events
1
, ,
are also independent. Thus the notion of

independence is symmetric in the events involved.

(iv) Events in any subcollection of independent events are independent. In
particular independence of a collection of events implies their pairwise
independence.



The following example illustrates that, in general, pairwise independence of a collection
of events may not imply their independence.

Example 3.6

Let = 1, 2, 3, 4 and let = , the power set of . Consider the probability space
, , P, where =
1
4
, = 1, 2, 3, 4 . Let = 1, 4, = 2, 4 and = 3, 4.
Then,
= = =
1
2
,
= = = {4} =
1
4
,
and = 4 =
1
4

Clearly,
= ; = (), and = (),
i.e., , and are pairwise independent.
However,
=
1
4
().
Thus , and are not independent.

Probability Topics: 1.3.2 Bayes' Theorem

Uploaded by

Copyright:

Available Formats

Probability Topics: 1.3.2 Bayes' Theorem

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Probability Topics: 1.3.2 Bayes' Theorem

Uploaded by

Copyright:

Available Formats

NPTEL- Probability and Distributions

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 1

> 0, . Then, for any event

using Theorem of Total Probability.

, , causes the occurrence of an event . Given that the

, , are referred to as prior

|, , are referred to as posterior

: are said to be pairwise independent if any pair of events

: are independent. i.e., if

are said to be independent if,

are independent one must verify

are independent then, for any permutation

are also independent. Thus the notion of

You might also like

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.