Yates & Goodman 3e

19

Figure 1.1

B1

B2

B3

B4

C1

C2

C3

C4

In this example of Theorem 1.8, the partition is B = {B1, B2, B3, B4} and
Ci = A\Bi for i = 1, . . . , 4. It should be apparent that A = C1 [C2 [C3 [C4.

Yates & Goodman 3e

Theorem 1.10

Law of Total Probability

For a partition {B1, B2, . . . , Bm} with P[Bi] > 0 for all i,
P [A] =

m
X

i=1

P [A|Bi] P [Bi] .

21

Yates & Goodman 3e

21

Example 1.19 Problem

A company has three machines B1, B2, and B3 making 1 k resistors.
Resistors within 50 of the nominal value are considered acceptable. It
has been observed that 80% of the resistors produced by B1 and 90% of
the resistors produced by B2 are acceptable. The percentage for machine
B3 is 60%. Each hour, machine B1 produces 3000 resistors, B2 produces
4000 resistors, and B3 produces 3000 resistors. All of the resistors are
mixed together at random in one bin and packed for shipment. What is
the probability that the company ships an acceptable resistor?

Yates & Goodman 3e

21

Example 1.19 Solution

Let A = {resistor is acceptable}. Using the resistor accuracy information
to formulate a probability model, we write
P [A|B1] = 0.8,

P [A|B2] = 0.9,

P [A|B3] = 0.6.

(1)

The production figures state that 3000 + 4000 + 3000 = 10,000 resistors per hour are produced. The fraction from machine B1 is P[B1] =
3000/10,000 = 0.3. Similarly, P[B2] = 0.4 and P[B3] = 0.3. Now it is a
simple matter to apply the law of total probability to find the acceptable
probability for all resistors shipped by the company:
P [A] = P [A|B1] P [B1] + P [A|B2] P [B2] + P [A|B3] P [B3]

(2)

= (0.8)(0.3) + (0.9)(0.4) + (0.6)(0.3) = 0.78.

(3)

For the whole factory, 78% of resistors are within 50 of the nominal
value.

Yates & Goodman 3e

23

Example 1.20 Problem

In Example 1.19 about a shipment of resistors from the factory, we learned
that:

The probability that a resistor is from machine B3 is P[B3] = 0.3.

The probability that a resistor is acceptable i.e., within 50 of
the nominal value is P[A] = 0.78.

Given that a resistor is from machine B3, the conditional probability

that it is acceptable is P[A|B3] = 0.6.
What is the probability that an acceptable resistor comes from machine
B3 ?

Yates & Goodman 3e

Theorem 1.11

Bayes theorem
P [B|A] =

P [A|B] P [B]
.
P [A]

22

Yates & Goodman 3e

23

Example 1.20 Solution

Now we are given the event A that a resistor is within 50 of the nominal
value, and we need to find P[B3|A]. Using Bayes theorem, we have
P [B3|A] =

P [A|B3] P [B3]
.
P [A]

(1)

Since all of the quantities we need are given in the problem description,
our answer is
P [B3|A] = (0.6)(0.3)/(0.78) = 0.23.

(2)

Similarly we obtain P[B1|A] = 0.31 and P[B2|A] = 0.46. Of all resistors

within 50 of the nominal value, only 23% come from machine B3
(even though this machine produces 30% of all resistors). Machine B1
produces 31% of the resistors that meet the 50 criterion and machine
B2 produces 46% of them.

Ques%ons(from(the((e)mailbag(
ECE(226(
Probability(and(Stochas%c(Processes(

P[Intersec%on]??(
(Anyways,(I'm(having(trouble(understanding(how(to(calculate(the(
probability(of(the(intersec%on(of(two(events.(For(example,(assuming(
there(are(6(circles(and(6(squares,(and(one(of(each(shape(is(colored,(the(
probability(of(picking(a(colored(circle(is(
(
P[circle](*(P[colored](=((1/2)(*((1/6)(=(1/12.((
However,(in(another(case(where(there(are(the(same(numbers(of(each(
shapes,(but(two(circles(are(colored((and(no(squares(are(colored),(the(
probability(of(picking(a(colored(circle(is(not(
(
P[circle](*(P[colored](=((1/2)(*((2/12)(=(1/12,((
(
but(is(2/12,(by(logic.(

assuming(there(are(6(circles(and(6(squares,(and(one(of(each(
shape(is(colored,(the(probability(of(picking(a(colored(circle(is(
(
P[circle](*(P[colored](=((1/2)(*((1/6)(=(1/12.((
(

P[circle](=(1/2(
P[colored](=(2/12((
P[circle,colored]=2/12(

However,(in(another(case(where(there(are(the(same(numbers(of(
each(shapes,(but(two(circles(are(colored((and(no(squares(are(
colored),(the(probability(of(picking(a(colored(circle(is((
NOT$P[circle](*(P[colored](=((1/2)(*((2/12)(=(1/12(

P[circle](=(1/2(
P[colored](=(2/12((
P[circle,colored]=2/12(

Yates & Goodman 3e

Section 1.6

Independence

24

Yates & Goodman 3e

Definition 1.6 Two Independent Events

Events A and B are independent if and only if
P [AB] = P [A] P [B] .

24

assuming(there(are(6(circles(and(6(squares,(and(one(of(each(
shape(is(colored,(the(probability(of(picking(a(colored(circle(is(
(
P[circle](*(P[colored](=((1/2)(*((1/6)(=(1/12.((
(

P[circle](=(1/2(
P[colored](=(2/12((
P[circle,colored]=2/12(

However,(in(another(case(where(there(are(the(same(numbers(of(
each(shapes,(but(two(circles(are(colored((and(no(squares(are(
colored),(the(probability(of(picking(a(colored(circle(is((
NOT$P[circle](*(P[colored](=((1/2)(*((2/12)(=(1/12(

P[circle](=(1/2(
P[colored](=(2/12((
P[circle,colored]=2/12(

Yates & Goodman 3e

24

Independent vs. Mutually

1.6 Comment: Exclusive
Keep in mind that independent and mutually exclusive are not synonyms.
In some contexts these words can have similar meanings, but this is
not the case in probability. Mutually exclusive events A and B have
no outcomes in common and therefore P[AB] = 0. In most situations
independent events are not mutually exclusive! Exceptions occur only
when P[A] = 0 or P[B] = 0. When we have to calculate probabilities,
knowledge that events A and B are mutually exclusive is very helpful.
Axiom 3 enables us to add their probabilities to obtain the probability of
the union. Knowledge that events C and D are independent is also very
useful. Definition 1.6 enables us to multiply their probabilities to obtain
the probability of the intersection.

Yates & Goodman 3e

24

Example 1.21 Problem

Suppose that for the experiment monitoring three purchasing decisions in
Example 1.9, each outcome (a sequence of three decisions, each either
buy or not buy) is equally likely. Are the events B2 that the second
customer purchases a phone and N2 that the second customer does not
purchase a phone independent? Are the events B1 and B2 independent?

Yates & Goodman 3e

24

Example 1.21 Solution

Each element of the sample space S = {bbb, bbn, bnb, bnn, nbb, nbn, nnb, nnn} has probability 1/8. Each of the events
B2 = {bbb, bbn, nbb, nbn}

and

N2 = {bnb, bnn, nnb, nnn}

(1)

contains four outcomes, so P[B2 ] = P[N2 ] = 4/8. However, B2 \ N2 = ? and P[B2 N2 ] =

0. That is, B2 and N2 are mutually exclusive because the second customer cannot both
purchase a phone and not purchase a phone. Since P[B2 N2 ] 6= P[B2 ] P[N2 ], B2 and N2
are not independent. Learning whether or not the event B2 (second customer buys a
phone) occurs drastically aects our knowledge of whether or not the event N2 (second
customer does not buy a phone) occurs. Each of the events B1 = {bnn, bnb, bbn, bbb}
and B2 = {bbn, bbb, nbn, nbb} has four outcomes, so P[B1 ] = P[B2 ] = 4/8 = 1/2. In this
case, the intersection B1 \ B2 = {bbn, bbb} has probability P[B1 B2 ] = 2/8 = 1/4. Since
P[B1 B2 ] = P[B1 ] P[B2 ], events B1 and B2 are independent. Learning whether or not
the event B2 (second customer buys a phone) occurs does not aect our knowledge of
whether or not the event B1 (first customer buys a phone) occurs.

Yates & Goodman 3e

25

Example 1.22 Problem

Integrated circuits undergo two tests. A mechanical test determines
whether pins have the correct spacing, and an electrical test checks the
relationship of outputs to inputs. We assume that electrical failures and
mechanical failures occur independently. Our information about circuit
production tells us that mechanical failures occur with probability 0.05
and electrical failures occur with probability 0.2. What is the probability
model of an experiment that consists of testing an integrated circuit and
observing the results of the mechanical and electrical tests?

Yates & Goodman 3e

25

Example 1.22 Solution

To build the probability model, we note that the sample space contains
four outcomes:
S = {(ma, ea), (ma, er), (mr, ea), (mr, er)}

(1)

where m denotes mechanical, e denotes electrical, a denotes accept, and

r denotes reject. Let M and E denote the events that the mechanical
and electrical tests are acceptable. Our prior information tells us that
P[M c] = 0.05, and P[E c] = 0.2. This implies P[M ] = 0.95 and P[E] = 0.8.
Using the independence assumption and Definition 1.6, we obtain the
probabilities of the four outcomes:
P [(ma, ea)] = P [M E] = P [M ] P [E] = 0.95 0.8 = 0.76,

(2)

P [(mr, ea)] = P [M cE] = P [M c] P [E] = 0.05 0.8 = 0.04,

(4)

P [(ma, er)] = P [M E c] = P [M ] P [E c] = 0.95 0.2 = 0.19,

(3)

P [(mr, er)] = P [M cE c] = P [M c] P [E c] = 0.05 0.2 = 0.01.

(5)

Yates & Goodman 3e

Definition 1.7 Three Independent Events

A1, A2, and A3 are mutually independent if and only if
(a) A1 and A2 are independent,
(b) A2 and A3 are independent,
(c) A1 and A3 are independent,
(d) P[A1 \ A2 \ A3] = P[A1] P[A2] P[A3].

26

Yates & Goodman 3e

26

Example 1.23 Problem

In an experiment with equiprobable outcomes, the partition is S = {1, 2, 3, 4}.
P[s] = 1/4 for all s 2 S. Are the events A1 = {1, 3, 4}, A2 = {2, 3, 4}, and
A3 = ? mutually independent?

Yates & Goodman 3e

26

Example 1.23 Solution

These three sets satisfy the final condition of Definition 1.7 because
A1 \ A2 \ A3 = ?, and
P [A1 \ A2 \ A3] = P [A1] P [A2] P [A3] = 0.

(1)

However, A1 and A2 are not independent because, with all outcomes

equiprobable,
P [A1 \ A2] = P [{3, 4}] = 1/2 6= P [A1] P [A2] = 3/4 3/4.
Hence the three events are not mutually independent.

(2)

Yates & Goodman 3e

26

More than Two Independent

Definition 1.8 Events
If n

3, the events A1, A2, . . . , An are mutually independent if and only if

(a) all collections of n

independent,

1 events chosen from A1, A2, . . . An are mutually

(b) P[A1 \ A2 \ \ An] = P[A1] P[A2] P[An].

Yates & Goodman 3e

27

Quiz 1.6
Monitor two consecutive packets going through a router. Classify each one as video (v)
if it was sent from a Youtube server or as ordinary data (d) otherwise. Your observation
is a sequence of two letters (either v or d). For example, two video packets corresponds
to vv. The two packets are independent and the probability that any one of them is
a video packet is 0.8. Denote the identity of packet i by Ci . If packet i is a video
packet, then Ci = v; otherwise, Ci = d. Count the number NV of video packets in the
two packets you have observed. Determine whether the following pairs of events are
independent:
(a) {NV = 2} and {NV
(b) {NV

1}

1} and {C1 = v}

(c) {C2 = v} and {C1 = d}

(d) {C2 = v} and {NV is even}

Yates & Goodman 3e

27

Quiz 1.6 Solution

In this experiment, there are four outcomes with probabilities
P[{vv}] = (0.8)2 = 0.64,

P[{vd}] = (0.8)(0.2) = 0.16,

P[{dv}] = (0.2)(0.8) = 0.16,

P[{dd}] = (0.2)2 = 0.04.

When checking the independence of any two events A and B, its wise to avoid intuition
and simply check whether P[AB] = P[A] P[B]. Using the probabilities of the outcomes,
we now can test for the independence of events.
(a) First, we calculate the probability of the joint event:
P [NV = 2, NV
Next, we observe that P[NV
comparison
P [NV = 2] P [NV

1] = P [NV = 2] = P [{vv}] = 0.64.

(1)

1] = P[{vd, dv, vv}] = 0.96.. Finally, we make the

1] = (0.64)(0.96) 6= P [NV = 2, NV

which shows the two events are dependent.

1] ,

(2)
[Continued]

Yates & Goodman 3e

Quiz 1.6 Solution

27

(Continued 2)

(b) The probability of the joint event is

P [NV
From part (a), P[NV
P [NV

1, C1 = v] = P [{vd, vv}] = 0.80.

(3)

1] = 0.96. Further, P[C1 = v] = 0.8 so that

1] P [C1 = v] = (0.96)(0.8) = 0.768 6= P [NV

1, C1 = v] .

(4)

Hence, the events are dependent.

(c) The problem statement that the packets were independent implies that the events
{C2 = v} and {C1 = d} are independent events. Just to be sure, we can do the
calculations to check:
P [C1 = d, C2 = v] = P [{dv}] = 0.16.

(5)

Since P[C1 = d] P[C2 = v] = (0.2)(0.8) = 0.16, we confirm that the events are
independent. Note that this shouldnt be surprising since we used the information
that the packets were independent in the problem statement to determine the
probabilities of the outcomes.
[Continued]

Yates & Goodman 3e

Quiz 1.6 Solution

27

(Continued 3)

(d) The probability of the joint event is

P [C2 = v, NV is even] = P [{vv}] = 0.64.

(6)

Also, each event has probability

P [C2 = v] = P [{dv, vv}] = 0.8,
P [NV is even] = P [{dd, vv}] = 0.68.

(7)
(8)

P [C2 = v] P [NV is even] = (0.8)(0.68)

= 0.544 6= P [C2 = v, NV is even] .

(9)

Thus,

Thus the events are dependent.

Yates & Goodman 3e

Section 1.7

Matlab

27

Yates & Goodman 3e

28

Example 1.24

>> X=rand(1,4)
X =
0.0879 0.9626 0.6627
>> X<0.5
ans =
1
0
0
1

0.2023

{head} and 0 with {tail}.

Since rand(1,4) < 0.5 compares four random numbers against 0.5, the result is a
random sequence of zeros and ones that
simulates a sequence of four flips of a fair
coin. We associate the outcome 1 with

Yates & Goodman 3e

28

Example 1.25 Problem

Use Matlab to generate 12 random student test scores T as described in
Quiz 1.3.

Yates & Goodman 3e

28

Example 1.25 Solution

Since randi(50,1,12) generates 12 test scores from the set {1, . . . , 50},
we need only to add 50 to each score to obtain test scores in the range
{51, . . . , 100}.
>> 50+randi(50,1,12)
ans =
69
78
60
68

93

99

77

95

88

57

51

90

Yates & Goodman 3e

Example 1.26
>> s=rng;
>> 50+randi(50,1,12)
ans =
89
76
80
80
>> rng(s);
>> 50+randi(50,1,12)
ans =
89
76
80
80

72

92

58

56

77

78

59

58

72

92

58

56

77

78

59

58

29

Yates & Goodman 3e

29

Quiz 1.7
The number of characters in a tweet is equally likely to be any integer
between 1 and 140. Simulate an experiment that generates 1000 tweets
and counts the number of long tweets that have over 120 characters.
Repeat this experiment 5 times.

Yates & Goodman 3e

29

Quiz 1.7 Solution

These two matlab instructions
>> T=randi(140,1000,5);
>> sum(T>120)
ans =
126
147
134
133

163

simulate 5 runs of an experiment each with 1000 tweets. In particular, we note that
T=randi(140,1000,5) generates a 1000 5 array T of pseudorandom integers between
1 and 140. Each column of T has 1000 entries representing an experimental run
corresponding to the lengths of 1000 tweets. The comparison T>120 produces a 51000
binary matrix in which each 1 marks a long tweet with length over 120 characters.
Summing this binary array along the columns with the command sum(T>120) counts the
number of long tweets in each experimental run.
The experiment in which we examine the length of one tweet has sample space S =
{s1 , s2 , . . . , s140 } with si denoting the outcome that a tweet has length i. Note that
P[si ] = 1/140 and thus
20
1
= .
(1)
140
7
Thus in each run of 1000 tweets, we would expect to see about 1/7 of the tweets, or
about 143 tweets, to be be long tweets with length of over 120 characters. However,
because the lengths are random, we see that we observe in the neighborhood of 143
long tweets in each run.
P [tweet length > 120] = P [{s121 , s122 , . . . , s140 }] =

PROBABILITY AND STOCHASTIC PROCESSES

A FRIENDLY INTRODUCTION FOR ELECTRICAL AND COMPUTER ENGINEERS

Third Edition

Chapter 2 Viewgraphs

Roy D. Yates & David J. Goodman

Yates & Goodman 3e

Section 2.1

Tree Diagrams

35

Yates & Goodman 3e

36

Example 2.1 Problem

For the resistors of Example 1.19, we used A to denote the event that
a randomly chosen resistor is within 50 of the nominal value. This
could mean acceptable. We use the notation N (not acceptable) for
the complement of A. The experiment of testing a resistor can be viewed
as a two-step procedure. First we identify which machine (B1, B2, or B3)
produced the resistor. Second, we find out if the resistor is acceptable.
Draw a tree for this sequential experiment. What is the probability of
choosing a resistor from machine B2 that is not acceptable?

Yates & Goodman 3e

36

Example 2.1 Solution

This two-step procedure is shown

B1
in the tree on the left. To use the
0.3

tree to find the probability of the

0.4

B2 X
HH
XXX
X
HH
XX
event B2N , a nonacceptable re0.04
H
0.1 X N B2 N
HH
0.6
H
0.3
B3 XXXX
0.18
A B3 A
sistor from machine B2, we start
XXX
X
0.12
N B3 N
0.4
at the left and find that the probability of reaching B2 is P[B2] = 0.4. We then move to the right to B2N
and multiply P[B2] by P[N |B2] = 0.1 to obtain P[B2N ] = (0.4)(0.1) =
0.04.
A
B1 A
0.8

N B1 N
0.2

0.9
A B2 A

0.24
0.06
0.36