Chapter 3: Probability

OpenIntro Statistics, 4th Edition

Slides developed by Mine Çetinkaya-Rundel of OpenIntro.

Defining probability
Random processes

• A random process is a
situation in which we know
what outcomes could
happen, but we don’t know
which particular outcome
will happen.
• Examples: coin tosses, die
rolls, iTunes shuffle, whether
the stock market goes up or
down tomorrow, etc.
• It can be helpful to model a http:// www.cnet.com.au/

process as random even if it itunes-just-how-random-is-random-339274094.htm

is not truly random.

1
Probability

• There are several possible interpretations of probability but

they (almost) completely agree on the mathematical rules
probability must follow.
• P(A) = Probability of event A
• 0 ≤ P(A) ≤ 1

2
Probability

• There are several possible interpretations of probability but

they (almost) completely agree on the mathematical rules
probability must follow.
• P(A) = Probability of event A
• 0 ≤ P(A) ≤ 1
• Frequentist interpretation:
• The probability of an outcome is the proportion of times the
outcome would occur if we observed the random process an
infinite number of times.

2
Probability

• There are several possible interpretations of probability but

they (almost) completely agree on the mathematical rules
probability must follow.
• P(A) = Probability of event A
• 0 ≤ P(A) ≤ 1
• Frequentist interpretation:
• The probability of an outcome is the proportion of times the
outcome would occur if we observed the random process an
infinite number of times.
• Bayesian interpretation:
• A Bayesian interprets probability as a subjective degree of
belief: For the same event, two separate people could have
different viewpoints and so assign different probabilities.
• Largely popularized by revolutionary advance in computational
technology and methods during the last twenty years.
2
Practice

Which of the following events would you be most surprised by?

(a) exactly 3 heads in 10 coin flips

(b) exactly 3 heads in 100 coin flips
(c) exactly 3 heads in 1000 coin flips

3
Practice

Which of the following events would you be most surprised by?

(a) exactly 3 heads in 10 coin flips

(b) exactly 3 heads in 100 coin flips
(c) exactly 3 heads in 1000 coin flips

3
Law of large numbers

Law of large numbers states that as more observations are

collected, the proportion of occurrences with a particular outcome,
p̂n , converges to the probability of that outcome, p.

4
Law of large numbers (cont.)

When tossing a fair coin, if heads comes up on each of the first 10

tosses, what do you think the chance is that another head will come
up on the next toss? 0.5, less than 0.5, or more than 0.5?

HHHHHHHHHH?

5
Law of large numbers (cont.)

When tossing a fair coin, if heads comes up on each of the first 10

tosses, what do you think the chance is that another head will come
up on the next toss? 0.5, less than 0.5, or more than 0.5?

HHHHHHHHHH?

• The probability is still 0.5, or there is still a 50% chance that

another head will come up on the next toss.
P(H on 11th toss) = P(T on 11th toss) = 0.5

5
Law of large numbers (cont.)

When tossing a fair coin, if heads comes up on each of the first 10

tosses, what do you think the chance is that another head will come
up on the next toss? 0.5, less than 0.5, or more than 0.5?

HHHHHHHHHH?

• The probability is still 0.5, or there is still a 50% chance that

another head will come up on the next toss.
P(H on 11th toss) = P(T on 11th toss) = 0.5
• The coin is not “due” for a tail.

5
Law of large numbers (cont.)

When tossing a fair coin, if heads comes up on each of the first 10

tosses, what do you think the chance is that another head will come
up on the next toss? 0.5, less than 0.5, or more than 0.5?

HHHHHHHHHH?

• The probability is still 0.5, or there is still a 50% chance that

another head will come up on the next toss.
P(H on 11th toss) = P(T on 11th toss) = 0.5
• The coin is not “due” for a tail.
• The common misunderstanding of the LLN is that random
processes are supposed to compensate for whatever
happened in the past; this is just not true and is also called
5
gambler’s fallacy (or law of averages).
Disjoint and non-disjoint outcomes

Disjoint (mutually exclusive) outcomes: Cannot happen at the

same time.

• The outcome of a single coin toss cannot be a head and a tail.

• A student both cannot fail and pass a class.
• A single card drawn from a deck cannot be an ace and a
queen.

6
Disjoint and non-disjoint outcomes

Disjoint (mutually exclusive) outcomes: Cannot happen at the

same time.

• The outcome of a single coin toss cannot be a head and a tail.

• A student both cannot fail and pass a class.
• A single card drawn from a deck cannot be an ace and a
queen.

Non-disjoint outcomes: Can happen at the same time.

• A student can get an A in Stats and A in Econ in the same

semester.

6
Union of non-disjoint events

What is the probability of drawing a jack or a red card from a well

shuffled full deck?

Figure from http:// www.milefoot.com/ math/ discrete/ counting/ cardfreq.htm. 7

Union of non-disjoint events

What is the probability of drawing a jack or a red card from a well

shuffled full deck?

P(jack or red) = P(jack) + P(red) − P(jack and red)

4 26 2 28
= + − =
52 52 52 52

Figure from http:// www.milefoot.com/ math/ discrete/ counting/ cardfreq.htm. 7

Practice

What is the probability that a randomly sampled student thinks mar-

ijuana should be legalized or they agree with their parents’ political
views?

Share Parents’ Politics

Legalize MJ No Yes Total
No 11 40 51
Yes 36 78 114
Total 47 118 165

40+36−78
(a) 165
114+118−78
(b) 165
78
(c) 165
78
(d) 188
11
(e) 47
8
Practice

What is the probability that a randomly sampled student thinks mar-

ijuana should be legalized or they agree with their parents’ political
views?

Share Parents’ Politics

Legalize MJ No Yes Total
No 11 40 51
Yes 36 78 114
Total 47 118 165

40+36−78
(a) 165
114+118−78
(b) 165
78
(c) 165
78
(d) 188
11
(e) 47
8
Recap

General addition rule

P(A or B) = P(A) + P(B) − P(A and B)

Note: For disjoint events P(A and B) = 0, so the above formula simplifies to
P(A or B) = P(A) + P(B).

9
Probability distributions

A probability distribution lists all possible events and the

probabilities with which they occur.

• The probability distribution for the gender of one kid:

Event Male Female
Probability 0.5 0.5

10
Probability distributions

A probability distribution lists all possible events and the

probabilities with which they occur.

• The probability distribution for the gender of one kid:

Event Male Female
Probability 0.5 0.5
• Rules for probability distributions:
1. The events listed must be disjoint
2. Each probability must be between 0 and 1
3. The probabilities must total 1

10
Probability distributions

A probability distribution lists all possible events and the

probabilities with which they occur.

• The probability distribution for the gender of one kid:

Event Male Female
Probability 0.5 0.5
• Rules for probability distributions:
1. The events listed must be disjoint
2. Each probability must be between 0 and 1
3. The probabilities must total 1
• The probability distribution for the genders of two kids:
Event MM FF MF FM
Probability 0.25 0.25 0.25 0.25

10
Practice

In a survey, 52% of respondents said they are Democrats. What is

the probability that a randomly selected respondent from this sam-
ple is a Republican?

(a) 0.48
(b) more than 0.48
(c) less than 0.48
(d) cannot calculate using only the information given

11
Practice

In a survey, 52% of respondents said they are Democrats. What is

the probability that a randomly selected respondent from this sam-
ple is a Republican?

(a) 0.48
(b) more than 0.48
(c) less than 0.48
(d) cannot calculate using only the information given

If the only two political parties are Republican and Democrat, then
(a) is possible. However it is also possible that some people do not
affiliate with a political party or affiliate with a party other than
these two. Then (c) is also possible. However (b) is definitely not
possible since it would result in the total probability for the sample
space being above 1.
11
Sample space and complements

Sample space is the collection of all possible outcomes of a trial.

• A couple has one kid, what is the sample space for the gender
of this kid? S = {M, F}
• A couple has two kids, what is the sample space for the
gender of these kids?

12
Sample space and complements

Sample space is the collection of all possible outcomes of a trial.

• A couple has one kid, what is the sample space for the gender
of this kid? S = {M, F}
• A couple has two kids, what is the sample space for the
gender of these kids? S = {MM, FF, FM, MF}

12
Sample space and complements

Sample space is the collection of all possible outcomes of a trial.

• A couple has one kid, what is the sample space for the gender
of this kid? S = {M, F}
• A couple has two kids, what is the sample space for the
gender of these kids? S = {MM, FF, FM, MF}

Complementary events are two mutually exclusive events whose

probabilities that add up to 1.

• A couple has one kid. If we know that the kid is not a boy,
what is gender of this kid? { M, F } → Boy and girl are
complementary outcomes.
• A couple has two kids, if we know that they are not both girls,
what are the possible gender combinations for these kids?
12
Sample space and complements

Sample space is the collection of all possible outcomes of a trial.

• A couple has one kid, what is the sample space for the gender
of this kid? S = {M, F}
• A couple has two kids, what is the sample space for the
gender of these kids? S = {MM, FF, FM, MF}

Complementary events are two mutually exclusive events whose

probabilities that add up to 1.

• A couple has one kid. If we know that the kid is not a boy,
what is gender of this kid? { M, F } → Boy and girl are
complementary outcomes.
• A couple has two kids, if we know that they are not both girls,
what are the possible gender combinations for these kids? {
MM, FF, FM, MF } 12
Independence

Two processes are independent if knowing the outcome of one

provides no useful information about the outcome of the other.

13
Independence

Two processes are independent if knowing the outcome of one

provides no useful information about the outcome of the other.

• Knowing that the coin landed on a head on the first toss

does not provide any useful information for determining what
the coin will land on in the second toss. → Outcomes of two
tosses of a coin are independent.

13
Independence

Two processes are independent if knowing the outcome of one

provides no useful information about the outcome of the other.

• Knowing that the coin landed on a head on the first toss

does not provide any useful information for determining what
the coin will land on in the second toss. → Outcomes of two
tosses of a coin are independent.
• Knowing that the first card drawn from a deck is an ace does
provide useful information for determining the probability of
drawing an ace in the second draw. → Outcomes of two draws
from a deck of cards (without replacement) are dependent.

13
Practice

Between January 9-12, 2013, SurveyUSA interviewed a random sample

of 500 NC residents asking them whether they think widespread gun own-
ership protects law abiding citizens from crime, or makes society more
dangerous. 58% of all respondents said it protects citizens. 67% of White
respondents, 28% of Black respondents, and 64% of Hispanic respon-
dents shared this view. Which of the below is true?

Opinion on gun ownership and race ethnicity are most likely

(a) complementary
(b) mutually exclusive
(c) independent
(d) dependent
(e) disjoint

http:// www.surveyusa.com/ client/ PollReport.aspx?g=a5f460ef-bba9-484b-8579-1101ea26421b

14
Practice

Between January 9-12, 2013, SurveyUSA interviewed a random sample

of 500 NC residents asking them whether they think widespread gun own-
ership protects law abiding citizens from crime, or makes society more
dangerous. 58% of all respondents said it protects citizens. 67% of White
respondents, 28% of Black respondents, and 64% of Hispanic respon-
dents shared this view. Which of the below is true?

Opinion on gun ownership and race ethnicity are most likely

(a) complementary
(b) mutually exclusive
(c) independent
(d) dependent
(e) disjoint

http:// www.surveyusa.com/ client/ PollReport.aspx?g=a5f460ef-bba9-484b-8579-1101ea26421b

14
Checking for independence

If P(A occurs, given that B is true) = P(A | B) = P(A), then A and B

are independent.

15
Checking for independence

If P(A occurs, given that B is true) = P(A | B) = P(A), then A and B

are independent.

P(protects citizens) = 0.58

15
Checking for independence

If P(A occurs, given that B is true) = P(A | B) = P(A), then A and B

are independent.

P(protects citizens) = 0.58

P(randomly selected NC resident says gun ownership protects

citizens, given that the resident is white) =
P(protects citizens | White) = 0.67

P(protects citizens | Black) = 0.28

P(protects citizens | Hispanic) = 0.64

15
Checking for independence

If P(A occurs, given that B is true) = P(A | B) = P(A), then A and B

are independent.

P(protects citizens) = 0.58

P(randomly selected NC resident says gun ownership protects

citizens, given that the resident is white) =
P(protects citizens | White) = 0.67

P(protects citizens | Black) = 0.28

P(protects citizens | Hispanic) = 0.64

P(protects citizens) varies by race/ethnicity, therefore opinion on

15
gun ownership and race ethnicity are most likely dependent.
Determining dependence based on sample data

• If conditional probabilities calculated based on sample data

suggest dependence between two variables, the next step is
to conduct a hypothesis test to determine if the observed
difference between the probabilities is likely or unlikely to have
happened by chance.
• If the observed difference between the conditional
probabilities is large, then there is stronger evidence that the
difference is real.
• If a sample is large, then even a small difference can provide
strong evidence of a real difference.

16
Determining dependence based on sample data

• If conditional probabilities calculated based on sample data

suggest dependence between two variables, the next step is
to conduct a hypothesis test to determine if the observed
difference between the probabilities is likely or unlikely to have
happened by chance.
• If the observed difference between the conditional
probabilities is large, then there is stronger evidence that the
difference is real.
• If a sample is large, then even a small difference can provide
strong evidence of a real difference.
We saw that P(protects citizens | White) = 0.67 and P(protects citizens |
Hispanic) = 0.64. Under which condition would you be more convinced
of a real difference between the proportions of Whites and Hispanics who
think gun widespread gun ownership protects citizens? n = 500 or n = 16
Determining dependence based on sample data

• If conditional probabilities calculated based on sample data

suggest dependence between two variables, the next step is
to conduct a hypothesis test to determine if the observed
difference between the probabilities is likely or unlikely to have
happened by chance.
• If the observed difference between the conditional
probabilities is large, then there is stronger evidence that the
difference is real.
• If a sample is large, then even a small difference can provide
strong evidence of a real difference.
We saw that P(protects citizens | White) = 0.67 and P(protects citizens |
Hispanic) = 0.64. Under which condition would you be more convinced
of a real difference between the proportions of Whites and Hispanics who
think gun widespread gun ownership protects citizens? n = 500 or n = 16
Product rule for independent events

P(A and B) = P(A) × P(B)

Or more generally, P(A1 and · · · and Ak ) = P(A1 ) × · · · × P(Ak )

17
Product rule for independent events

P(A and B) = P(A) × P(B)

Or more generally, P(A1 and · · · and Ak ) = P(A1 ) × · · · × P(Ak )

You toss a coin twice, what is the probability of getting two tails in a
row?

17
Product rule for independent events

P(A and B) = P(A) × P(B)

Or more generally, P(A1 and · · · and Ak ) = P(A1 ) × · · · × P(Ak )

You toss a coin twice, what is the probability of getting two tails in a
row?

1 1 1
P(T on the first toss) × P(T on the second toss) = × =
2 2 4

17
Practice

A recent Gallup poll suggests that 25.5% of Texans do not have

health insurance as of June 2012. Assuming that the uninsured rate
stayed constant, what is the probability that two randomly selected
Texans are both uninsured?

(a) 25.52
(b) 0.2552
(c) 0.255 × 2
(d) (1 − 0.255)2

http:// www.gallup.com/ poll/ 156851/ uninsured-rate-stable-across-states-far-2012.aspx

18
Practice

A recent Gallup poll suggests that 25.5% of Texans do not have

health insurance as of June 2012. Assuming that the uninsured rate
stayed constant, what is the probability that two randomly selected
Texans are both uninsured?

(a) 25.52
(b) 0.2552
(c) 0.255 × 2
(d) (1 − 0.255)2

http:// www.gallup.com/ poll/ 156851/ uninsured-rate-stable-across-states-far-2012.aspx

18
Disjoint vs. complementary

Do the sum of probabilities of two disjoint events always add up to

1?

19
Disjoint vs. complementary

Do the sum of probabilities of two disjoint events always add up to

1?

Not necessarily, there may be more than 2 events in the sample

space, e.g. party affiliation.

19
Disjoint vs. complementary

Do the sum of probabilities of two disjoint events always add up to

1?

Not necessarily, there may be more than 2 events in the sample

space, e.g. party affiliation.

Do the sum of probabilities of two complementary events always

add up to 1?

19
Disjoint vs. complementary

Do the sum of probabilities of two disjoint events always add up to

1?

Not necessarily, there may be more than 2 events in the sample

space, e.g. party affiliation.

Do the sum of probabilities of two complementary events always

add up to 1?

Yes, that’s the definition of complementary, e.g. heads and tails.

19
Putting everything together...

If we were to randomly select 5 Texans, what is the probability that

at least one is uninsured?

• If we were to randomly select 5 Texans, the sample space for

the number of Texans who are uninsured would be:

S = {0, 1, 2, 3, 4, 5}

• We are interested in instances where at least one person is

uninsured:
S = {0, 1, 2, 3, 4, 5}
• So we can divide up the sample space into two categories:

S = {0, at least one}

20
Putting everything together...

Since the probability of the sample space must add up to 1:

Prob(at least 1 uninsured) = 1 − Prob(none uninsured)

21
Putting everything together...

Since the probability of the sample space must add up to 1:

Prob(at least 1 uninsured) = 1 − Prob(none uninsured)
= 1 − [(1 − 0.255)5 ]

21
Putting everything together...

Since the probability of the sample space must add up to 1:

Prob(at least 1 uninsured) = 1 − Prob(none uninsured)
= 1 − [(1 − 0.255)5 ]
= 1 − 0.7455

21
Putting everything together...

Since the probability of the sample space must add up to 1:

Prob(at least 1 uninsured) = 1 − Prob(none uninsured)
= 1 − [(1 − 0.255)5 ]
= 1 − 0.7455
= 1 − 0.23

21
Putting everything together...

Since the probability of the sample space must add up to 1:

Prob(at least 1 uninsured) = 1 − Prob(none uninsured)
= 1 − [(1 − 0.255)5 ]
= 1 − 0.7455
= 1 − 0.23
= 0.77

At least 1

P(at least one) = 1 − P(none)

21
Practice

Roughly 20% of undergraduates at a university are vegetarian or

vegan. What is the probability that, among a random sample of 3
undergraduates, at least one is vegetarian or vegan?

(a) 1 − 0.2 × 3
(b) 1 − 0.23
(c) 0.83
(d) 1 − 0.8 × 3
(e) 1 − 0.83

22
Practice

Roughly 20% of undergraduates at a university are vegetarian or

vegan. What is the probability that, among a random sample of 3
undergraduates, at least one is vegetarian or vegan?

(a) 1 − 0.2 × 3
P(at least 1 from veg) = 1 − P(none veg)
(b) 1 − 0.23
= 1 − (1 − 0.2)3
(c) 0.83 = 1 − 0.83
(d) 1 − 0.8 × 3 = 1 − 0.512 = 0.488
(e) 1 − 0.83

22
Conditional probability
Relapse

Researchers randomly assigned 72 chronic users of cocaine into

three groups: desipramine (antidepressant), lithium (standard
treatment for cocaine) and placebo. Results of the study are
summarized below.

no
relapse relapse total
desipramine 10 14 24
lithium 18 6 24
placebo 20 4 24
total 48 24 72

http:// www.oswego.edu/ ∼srp/ stats/ 2 way tbl 1.htm

23
What is the probability that a patient did not relapse?

no
relapse relapse total
desipramine 10 14 24
lithium 18 6 24
placebo 20 4 24
total 48 24 72

24
Marginal probability

What is the probability that a patient relapsed?

no
relapse relapse total
desipramine 10 14 24
lithium 18 6 24
placebo 20 4 24
total 48 24 72

25
Marginal probability

What is the probability that a patient relapsed?

no
relapse relapse total
desipramine 10 14 24
lithium 18 6 24
placebo 20 4 24
total 48 24 72

48
P(relapsed) = 72 ≈ 0.67

25
Joint probability

What is the probability that a patient received the antidepressant

(desipramine) and relapsed?

no
relapse relapse total
desipramine 10 14 24
lithium 18 6 24
placebo 20 4 24
total 48 24 72

26
Joint probability

What is the probability that a patient received the antidepressant

(desipramine) and relapsed?

no
relapse relapse total
desipramine 10 14 24
lithium 18 6 24
placebo 20 4 24
total 48 24 72

10
P(relapsed and desipramine) = 72 ≈ 0.14

26
Conditional probability

Conditional probability

The conditional probability of the outcome of interest A given

condition B is calculated as
P(A and B)
P(A|B) =
P(B)

27
Conditional probability

Conditional probability

The conditional probability of the outcome of interest A given

condition B is calculated as
P(A and B)
P(A|B) =
P(B)

P(relapse|desipramine)
no P(relapse and desipramine)
=
relapse relapse total P(desipramine)
desipramine 10 14 24
lithium 18 6 24
placebo 20 4 24
total 48 24 72
27
Conditional probability

Conditional probability

The conditional probability of the outcome of interest A given

condition B is calculated as
P(A and B)
P(A|B) =
P(B)

P(relapse|desipramine)
no P(relapse and desipramine)
=
relapse relapse total P(desipramine)
desipramine 10 14 24 10/72
=
lithium 18 6 24 24/72
placebo 20 4 24
total 48 24 72
27
Conditional probability

Conditional probability

The conditional probability of the outcome of interest A given

condition B is calculated as
P(A and B)
P(A|B) =
P(B)

P(relapse|desipramine)
no P(relapse and desipramine)
=
relapse relapse total P(desipramine)
desipramine 10 14 24 10/72
=
lithium 18 6 24 24/72
placebo 20 4 24 10
=
total 48 24 72 24
27
Conditional probability

Conditional probability

The conditional probability of the outcome of interest A given

condition B is calculated as
P(A and B)
P(A|B) =
P(B)

P(relapse|desipramine)
no P(relapse and desipramine)
=
relapse relapse total P(desipramine)
desipramine 10 14 24 10/72
=
lithium 18 6 24 24/72
placebo 20 4 24 10
=
total 48 24 72 24
= 0.42
27
Conditional probability (cont.)

If we know that a patient received the antidepressant (desipramine),

what is the probability that they relapsed?

no
relapse relapse total
desipramine 10 14 24
lithium 18 6 24
placebo 20 4 24
total 48 24 72

28
Conditional probability (cont.)

If we know that a patient received the antidepressant (desipramine),

what is the probability that they relapsed?

no
relapse relapse total
desipramine 10 14 24
lithium 18 6 24
placebo 20 4 24
total 48 24 72

10
P(relapse | desipramine) = 24 ≈ 0.42

28
Conditional probability (cont.)

If we know that a patient received the antidepressant (desipramine),

what is the probability that they relapsed?

no
relapse relapse total
desipramine 10 14 24
lithium 18 6 24
placebo 20 4 24
total 48 24 72

10
P(relapse | desipramine) = 24 ≈ 0.42

18
P(relapse | lithium) = 24 ≈ 0.75
P(relapse | placebo) = 20
24 ≈ 0.83

28
Conditional probability (cont.)

If we know that a patient relapsed, what is the probability that they

received the antidepressant (desipramine)?

no
relapse relapse total
desipramine 10 14 24
lithium 18 6 24
placebo 20 4 24
total 48 24 72

29
Conditional probability (cont.)

If we know that a patient relapsed, what is the probability that they

received the antidepressant (desipramine)?

no
relapse relapse total
desipramine 10 14 24
lithium 18 6 24
placebo 20 4 24
total 48 24 72

10
P(desipramine | relapse) = 48 ≈ 0.21

29
Conditional probability (cont.)

If we know that a patient relapsed, what is the probability that they

received the antidepressant (desipramine)?

no
relapse relapse total
desipramine 10 14 24
lithium 18 6 24
placebo 20 4 24
total 48 24 72

10
P(desipramine | relapse) = 48 ≈ 0.21

18
P(lithium | relapse) = 48 ≈ 0.375
P(placebo | relapse) = 20
48 ≈ 0.42

29
General multiplication rule

• Earlier we saw that if two events are independent, their joint

probability is simply the product of their probabilities. If the
events are not believed to be independent, the joint probability
is calculated slightly differently.

30
General multiplication rule

• Earlier we saw that if two events are independent, their joint

probability is simply the product of their probabilities. If the
events are not believed to be independent, the joint probability
is calculated slightly differently.
• If A and B represent two outcomes or events, then

P(A and B) = P(A|B) × P(B)

Note that this formula is simply the conditional probability
formula, rearranged.

30
General multiplication rule

• Earlier we saw that if two events are independent, their joint

probability is simply the product of their probabilities. If the
events are not believed to be independent, the joint probability
is calculated slightly differently.
• If A and B represent two outcomes or events, then

P(A and B) = P(A|B) × P(B)

Note that this formula is simply the conditional probability
formula, rearranged.
• It is useful to think of A as the outcome of interest and B as
the condition.

30
Independence and conditional probabilities

Consider the following (hypothetical) distribution of gender and

major of students in an introductory statistics class:

social non-social
science science total
female 30 20 50
male 30 20 50
total 60 40 100

31
Independence and conditional probabilities

Consider the following (hypothetical) distribution of gender and

major of students in an introductory statistics class:

social non-social
science science total
female 30 20 50
male 30 20 50
total 60 40 100

• The probability that a randomly selected student is a social

science major is

31
Independence and conditional probabilities

Consider the following (hypothetical) distribution of gender and

major of students in an introductory statistics class:

social non-social
science science total
female 30 20 50
male 30 20 50
total 60 40 100

• The probability that a randomly selected student is a social

science major is 60
100 = 0.6.

31
Independence and conditional probabilities

Consider the following (hypothetical) distribution of gender and

major of students in an introductory statistics class:

social non-social
science science total
female 30 20 50
male 30 20 50
total 60 40 100

• The probability that a randomly selected student is a social

science major is 60
100 = 0.6.
• The probability that a randomly selected student is a social
science major given that they are female is

31
Independence and conditional probabilities

Consider the following (hypothetical) distribution of gender and

major of students in an introductory statistics class:

social non-social
science science total
female 30 20 50
male 30 20 50
total 60 40 100

• The probability that a randomly selected student is a social

science major is 60
100 = 0.6.
• The probability that a randomly selected student is a social
science major given that they are female is 30
50 = 0.6.

31
Independence and conditional probabilities

Consider the following (hypothetical) distribution of gender and

major of students in an introductory statistics class:

social non-social
science science total
female 30 20 50
male 30 20 50
total 60 40 100

• The probability that a randomly selected student is a social

science major is 60
100 = 0.6.
• The probability that a randomly selected student is a social
science major given that they are female is 30
50 = 0.6.
• Since P(SS|M) also equals 0.6, major of students in this class
does not depend on their gender: P(SS | F) = P(SS).
31
Independence and conditional probabilities (cont.)

Generically, if P(A|B) = P(A) then the events A and B are said to be

independent.

32
Independence and conditional probabilities (cont.)

Generically, if P(A|B) = P(A) then the events A and B are said to be

independent.

• Conceptually: Giving B doesn’t tell us anything about A.

32
Independence and conditional probabilities (cont.)

Generically, if P(A|B) = P(A) then the events A and B are said to be

independent.

• Conceptually: Giving B doesn’t tell us anything about A.

• Mathematically: We know that if events A and B are
independent, P(A and B) = P(A) × P(B). Then,

P(A and B) P(A) × P(B)

P(A|B) = = = P(A)
P(B) P(B)

32
Breast cancer screening

• American Cancer Society estimates that about 1.7% of

women have breast cancer.
http:// www.cancer.org/ cancer/ cancerbasics/ cancer-prevalence
• Susan G. Komen For The Cure Foundation states that
mammography correctly identifies about 78% of women who
truly have breast cancer.
http:
// ww5.komen.org/ BreastCancer/ AccuracyofMammograms.html
• An article published in 2003 suggests that up to 10% of all
mammograms result in false positives for patients who do not
have cancer.
http:// www.ncbi.nlm.nih.gov/ pmc/ articles/ PMC1360940

Note: These percentages are approximate, and very difficult to estimate. 33

Inverting probabilities

When a patient goes through breast cancer screening there are two
competing claims: patient had cancer and patient doesn’t have can-
cer. If a mammogram yields a positive result, what is the probability
that patient actually has cancer?

34
Inverting probabilities

When a patient goes through breast cancer screening there are two
competing claims: patient had cancer and patient doesn’t have can-
cer. If a mammogram yields a positive result, what is the probability
that patient actually has cancer?
Cancer status Test result

positive, 0.78
0.017*0.78 = 0.0133
cancer, 0.017

negative, 0.22
0.017*0.22 = 0.0037

positive, 0.1
0.983*0.1 = 0.0983
no cancer, 0.983

negative, 0.9
0.983*0.9 = 0.8847

34
Inverting probabilities

When a patient goes through breast cancer screening there are two
competing claims: patient had cancer and patient doesn’t have can-
cer. If a mammogram yields a positive result, what is the probability
that patient actually has cancer?
Cancer status Test result

positive, 0.78
0.017*0.78 = 0.0133
cancer, 0.017
P(C|+)
negative, 0.22
0.017*0.22 = 0.0037

positive, 0.1
0.983*0.1 = 0.0983
no cancer, 0.983

negative, 0.9
0.983*0.9 = 0.8847

34
Inverting probabilities

When a patient goes through breast cancer screening there are two
competing claims: patient had cancer and patient doesn’t have can-
cer. If a mammogram yields a positive result, what is the probability
that patient actually has cancer?
Cancer status Test result

positive, 0.78
0.017*0.78 = 0.0133
cancer, 0.017
P(C|+)
P(C and +)
negative, 0.22
0.017*0.22 = 0.0037 =
P(+)
positive, 0.1
0.983*0.1 = 0.0983
no cancer, 0.983

negative, 0.9
0.983*0.9 = 0.8847

34
Inverting probabilities

When a patient goes through breast cancer screening there are two
competing claims: patient had cancer and patient doesn’t have can-
cer. If a mammogram yields a positive result, what is the probability
that patient actually has cancer?
Cancer status Test result

positive, 0.78
0.017*0.78 = 0.0133
cancer, 0.017
P(C|+)
P(C and +)
negative, 0.22
0.017*0.22 = 0.0037 =
P(+)
0.0133
positive, 0.1
0.983*0.1 = 0.0983 =
no cancer, 0.983
0.0133 + 0.0983
negative, 0.9
0.983*0.9 = 0.8847

34
Inverting probabilities

When a patient goes through breast cancer screening there are two
competing claims: patient had cancer and patient doesn’t have can-
cer. If a mammogram yields a positive result, what is the probability
that patient actually has cancer?
Cancer status Test result

positive, 0.78
0.017*0.78 = 0.0133
cancer, 0.017
P(C|+)
P(C and +)
negative, 0.22
0.017*0.22 = 0.0037 =
P(+)
0.0133
positive, 0.1
0.983*0.1 = 0.0983 =
no cancer, 0.983
0.0133 + 0.0983
= 0.12
negative, 0.9
0.983*0.9 = 0.8847

34
Inverting probabilities

When a patient goes through breast cancer screening there are two
competing claims: patient had cancer and patient doesn’t have can-
cer. If a mammogram yields a positive result, what is the probability
that patient actually has cancer?
Cancer status Test result

positive, 0.78
0.017*0.78 = 0.0133
cancer, 0.017
P(C|+)
P(C and +)
negative, 0.22
0.017*0.22 = 0.0037 =
P(+)
0.0133
positive, 0.1
0.983*0.1 = 0.0983 =
no cancer, 0.983
0.0133 + 0.0983
= 0.12
negative, 0.9
0.983*0.9 = 0.8847

Note: Tree diagrams are useful for inverting probabilities: we are given P(+|C)
34
and asked for P(C|+).
Practice

Suppose a woman who gets tested once and obtains a positive

result wants to get tested again. In the second test, what should we
assume to be the probability of this specific woman having cancer?

(a) 0.017
(b) 0.12
(c) 0.0133
(d) 0.88

35
Practice

Suppose a woman who gets tested once and obtains a positive

result wants to get tested again. In the second test, what should we
assume to be the probability of this specific woman having cancer?

(a) 0.017
(b) 0.12
(c) 0.0133
(d) 0.88

35
Practice

What is the probability that this woman has cancer if this second
mammogram also yielded a positive result?

(a) 0.0936
(b) 0.088
(c) 0.48
(d) 0.52

36
Practice

What is the probability that this woman has cancer if this second
mammogram also yielded a positive result?
Cancer status Test result

positive, 0.78
0.12*0.78 = 0.0936
cancer, 0.12
(a) 0.0936
negative, 0.22
0.12*0.22 = 0.0264
(b) 0.088
positive, 0.1
(c) 0.48 0.88*0.1 = 0.088
no cancer, 0.88

(d) 0.52 negative, 0.9

0.88*0.9 = 0.792

36
Practice

What is the probability that this woman has cancer if this second
mammogram also yielded a positive result?
Cancer status Test result

positive, 0.78
0.12*0.78 = 0.0936
cancer, 0.12
(a) 0.0936
negative, 0.22
0.12*0.22 = 0.0264
(b) 0.088
positive, 0.1
(c) 0.48 0.88*0.1 = 0.088
no cancer, 0.88

(d) 0.52 negative, 0.9

0.88*0.9 = 0.792

P(C and +) 0.0936

P(C|+) = = = 0.52
P(+) 0.0936 + 0.088

36
Bayes’ Theorem

• The conditional probability formula we have seen so far is a

special case of the Bayes’ Theorem, which is applicable even
when events have more than just two outcomes.

37
Bayes’ Theorem

• The conditional probability formula we have seen so far is a

special case of the Bayes’ Theorem, which is applicable even
when events have more than just two outcomes.
• Bayes’ Theorem:

P(outcome A1 of variable 1 | outcome B of variable 2)

P(B|A1 )P(A1 )
=
P(B|A1 )P(A1 ) + P(B|A2 )P(A2 ) + · · · + P(B|Ak )P(Ak )
where A2 , · · · , Ak represent all other possible outcomes of
variable 1.

37
Application activity: Inverting probabilities

A common epidemiological model for the spread of diseases is the SIR model,
where the population is partitioned into three groups: Susceptible, Infected, and
Recovered. This is a reasonable model for diseases like chickenpox where a
single infection usually provides immunity to subsequent infections. Sometimes
these diseases can also be difficult to detect.

Imagine a population in the midst of an epidemic where 60% of the population is

considered susceptible, 10% is infected, and 30% is recovered. The only test for
the disease is accurate 95% of the time for susceptible individuals, 99% for infected
individuals, but 65% for recovered individuals. (Note: In this case accurate means
returning a negative result for susceptible and recovered individuals and a positive
result for infected individuals).

Draw a probability tree to reflect the information given above. If the individual has
tested positive, what is the probability that they are actually infected?
38
Application activity: Inverting probabilities (cont.)
Group Test result

positive, 0.05
0.03
susceptible, 0.6
negative, 0.95
0.57

positive, 0.99
0.099
infected, 0.1
negative, 0.01
0.001

positive, 0.35
0.105
recovered, 0.3
negative, 0.65
0.195

39
Application activity: Inverting probabilities (cont.)
Group Test result

positive, 0.05
0.03
susceptible, 0.6
negative, 0.95
0.57

positive, 0.99
0.099
infected, 0.1
negative, 0.01
0.001

positive, 0.35
0.105
recovered, 0.3
negative, 0.65
0.195

P(inf and +) 0.099

P(inf |+) = = ≈ 0.423
P(+) 0.03 + 0.099 + 0.105
39
Sampling from a small population
Sampling with replacement

When sampling with replacement, you put back what you just drew.

40
Sampling with replacement

When sampling with replacement, you put back what you just drew.

• Imagine you have a bag with 5 red, 3 blue and 2 orange chips
in it. What is the probability that the first chip you draw is blue?
5 ,3 ,2

40
Sampling with replacement

When sampling with replacement, you put back what you just drew.

• Imagine you have a bag with 5 red, 3 blue and 2 orange chips
in it. What is the probability that the first chip you draw is blue?
5 ,3 ,2

3 3
Prob(1st chip B) = = = 0.3
5 + 3 + 2 10

40
Sampling with replacement

When sampling with replacement, you put back what you just drew.

• Imagine you have a bag with 5 red, 3 blue and 2 orange chips
in it. What is the probability that the first chip you draw is blue?
5 ,3 ,2

3 3
Prob(1st chip B) = = = 0.3
5 + 3 + 2 10
• Suppose you did indeed pull a blue chip in the first draw. If
drawing with replacement, what is the probability of drawing a
blue chip in the second draw?

40
Sampling with replacement

When sampling with replacement, you put back what you just drew.

• Imagine you have a bag with 5 red, 3 blue and 2 orange chips
in it. What is the probability that the first chip you draw is blue?
5 ,3 ,2

3 3
Prob(1st chip B) = = = 0.3
5 + 3 + 2 10
• Suppose you did indeed pull a blue chip in the first draw. If
drawing with replacement, what is the probability of drawing a
blue chip in the second draw?
1st draw: 5 ,3 ,2

40
Sampling with replacement

When sampling with replacement, you put back what you just drew.

• Imagine you have a bag with 5 red, 3 blue and 2 orange chips
in it. What is the probability that the first chip you draw is blue?
5 ,3 ,2

3 3
Prob(1st chip B) = = = 0.3
5 + 3 + 2 10
• Suppose you did indeed pull a blue chip in the first draw. If
drawing with replacement, what is the probability of drawing a
blue chip in the second draw?
1st draw: 5 ,3 ,2
2nd draw: 5 ,3 ,2

40
Sampling with replacement

When sampling with replacement, you put back what you just drew.

• Imagine you have a bag with 5 red, 3 blue and 2 orange chips
in it. What is the probability that the first chip you draw is blue?
5 ,3 ,2

3 3
Prob(1st chip B) = = = 0.3
5 + 3 + 2 10
• Suppose you did indeed pull a blue chip in the first draw. If
drawing with replacement, what is the probability of drawing a
blue chip in the second draw?
1st draw: 5 ,3 ,2
2nd draw: 5 ,3 ,2

3
Prob(2nd chip B|1st chip B) = = 0.3
10 40
Sampling with replacement (cont.)

• Suppose you actually pulled an orange chip in the first draw. If

drawing with replacement, what is the probability of drawing a
blue chip in the second draw?

41
Sampling with replacement (cont.)

• Suppose you actually pulled an orange chip in the first draw. If

drawing with replacement, what is the probability of drawing a
blue chip in the second draw?
1st draw: 5 ,3 ,2

41
Sampling with replacement (cont.)

• Suppose you actually pulled an orange chip in the first draw. If

drawing with replacement, what is the probability of drawing a
blue chip in the second draw?
1st draw: 5 ,3 ,2
2nd draw: 5 ,3 ,2

41
Sampling with replacement (cont.)

• Suppose you actually pulled an orange chip in the first draw. If

drawing with replacement, what is the probability of drawing a
blue chip in the second draw?
1st draw: 5 ,3 ,2
2nd draw: 5 ,3 ,2
3
Prob(2 nd
chip B|1 chip O) =
st
= 0.3
10

41
Sampling with replacement (cont.)

• Suppose you actually pulled an orange chip in the first draw. If

drawing with replacement, what is the probability of drawing a
blue chip in the second draw?
1st draw: 5 ,3 ,2
2nd draw: 5 ,3 ,2
3
Prob(2 nd
chip B|1 chip O) =
st
= 0.3
10
• If drawing with replacement, what is the probability of drawing
two blue chips in a row?

41
Sampling with replacement (cont.)

• Suppose you actually pulled an orange chip in the first draw. If

drawing with replacement, what is the probability of drawing a
blue chip in the second draw?
1st draw: 5 ,3 ,2
2nd draw: 5 ,3 ,2
3
Prob(2 nd
chip B|1 chip O) =
st
= 0.3
10
• If drawing with replacement, what is the probability of drawing
two blue chips in a row?
1st draw: 5 ,3 ,2
2nd draw: 5 ,3 ,2

41
Sampling with replacement (cont.)

• Suppose you actually pulled an orange chip in the first draw. If

drawing with replacement, what is the probability of drawing a
blue chip in the second draw?
1st draw: 5 ,3 ,2
2nd draw: 5 ,3 ,2
3
nd
Prob(2 chip B|1 chip O) =
st
= 0.3
10
• If drawing with replacement, what is the probability of drawing
two blue chips in a row?
1st draw: 5 ,3 ,2
2nd draw: 5 ,3 ,2
st
Prob(1 chip B) · Prob(2 nd
chip B|1 chip B) = 0.3 × 0.3
st

= 0.32 = 0.09
41
Sampling with replacement (cont.)

• When drawing with replacement, probability of the second

chip being blue does not depend on the color of the first chip
since whatever we draw in the first draw gets put back in the
bag.
Prob(B|B) = Prob(B|O)

• In addition, this probability is equal to the probability of

drawing a blue chip in the first draw, since the composition of
the bag never changes when sampling with replacement.

Prob(B|B) = Prob(B)

• When drawing with replacement, draws are independent.

42
Sampling without replacement

When drawing without replacement you do not put back what you
just drew.

43
Sampling without replacement

When drawing without replacement you do not put back what you
just drew.

• Suppose you pulled a blue chip in the first draw. If drawing

without replacement, what is the probability of drawing a blue
chip in the second draw?

43
Sampling without replacement

When drawing without replacement you do not put back what you
just drew.

• Suppose you pulled a blue chip in the first draw. If drawing

without replacement, what is the probability of drawing a blue
chip in the second draw?
1st draw: 5 ,3 ,2

43
Sampling without replacement

When drawing without replacement you do not put back what you
just drew.

• Suppose you pulled a blue chip in the first draw. If drawing

without replacement, what is the probability of drawing a blue
chip in the second draw?
1st draw: 5 ,3 ,2
2nd draw: 5 ,2 ,2

43
Sampling without replacement

When drawing without replacement you do not put back what you
just drew.

• Suppose you pulled a blue chip in the first draw. If drawing

without replacement, what is the probability of drawing a blue
chip in the second draw?
1st draw: 5 ,3 ,2
2nd draw: 5 ,2 ,2
2
Prob(2 nd
chip B|1 chip B) =
st
= 0.22
9

43
Sampling without replacement

When drawing without replacement you do not put back what you
just drew.

• Suppose you pulled a blue chip in the first draw. If drawing

without replacement, what is the probability of drawing a blue
chip in the second draw?
1st draw: 5 ,3 ,2
2nd draw: 5 ,2 ,2
2
Prob(2 nd
chip B|1 chip B) =
st
= 0.22
9
• If drawing without replacement, what is the probability of
drawing two blue chips in a row?

43
Sampling without replacement

When drawing without replacement you do not put back what you
just drew.

• Suppose you pulled a blue chip in the first draw. If drawing

without replacement, what is the probability of drawing a blue
chip in the second draw?
1st draw: 5 ,3 ,2
2nd draw: 5 ,2 ,2
2
Prob(2 nd
chip B|1 chip B) =
st
= 0.22
9
• If drawing without replacement, what is the probability of
drawing two blue chips in a row?
1st draw: 5 ,3 ,2
2nd draw: 5 ,2 ,2

43
Sampling without replacement

When drawing without replacement you do not put back what you
just drew.

• Suppose you pulled a blue chip in the first draw. If drawing

without replacement, what is the probability of drawing a blue
chip in the second draw?
1st draw: 5 ,3 ,2
2nd draw: 5 ,2 ,2
2
Prob(2 nd
chip B|1 chip B) =
st
= 0.22
9
• If drawing without replacement, what is the probability of
drawing two blue chips in a row?
1st draw: 5 , 3 , 2
2nd draw: 5 , 2 , 2
Prob(1st chip B) · Prob(2nd chip B|1st chip B) = 0.3 × 0.22
43
Sampling without replacement (cont.)

• When drawing without replacement, the probability of the

second chip being blue given the first was blue is not equal to
the probability of drawing a blue chip in the first draw since the
composition of the bag changes with the outcome of the first
draw.
Prob(B|B) , Prob(B)

44
Sampling without replacement (cont.)

• When drawing without replacement, the probability of the

second chip being blue given the first was blue is not equal to
the probability of drawing a blue chip in the first draw since the
composition of the bag changes with the outcome of the first
draw.
Prob(B|B) , Prob(B)

• When drawing without replacement, draws are not

independent.

44
Sampling without replacement (cont.)

• When drawing without replacement, the probability of the

second chip being blue given the first was blue is not equal to
the probability of drawing a blue chip in the first draw since the
composition of the bag changes with the outcome of the first
draw.
Prob(B|B) , Prob(B)

• When drawing without replacement, draws are not

independent.
• This is especially important to take note of when the sample
sizes are small. If we were dealing with, say, 10,000 chips in a
(giant) bag, taking out one chip of any color would not have as
big an impact on the probabilities in the second draw.

44
Practice

In most card games cards are dealt without replacement. What is

the probability of being dealt an ace and then a 3? Choose the
closest answer.

(a) 0.0045
(b) 0.0059
(c) 0.0060
(d) 0.1553

45
Practice

In most card games cards are dealt without replacement. What is

the probability of being dealt an ace and then a 3? Choose the
closest answer.

(a) 0.0045
4 4
(b) 0.0059 P(ace then 3) = × ≈ 0.0060
52 51
(c) 0.0060
(d) 0.1553

45
Random variables
Random variables

• A random variable is a numeric quantity whose value depends

on the outcome of a random event
• We use a capital letter, like X , to denote a random variable
• The values of a random variable are denoted with a lowercase
letter, in this case x
• For example, P(X = x)
• There are two types of random variables:
• Discrete random variables often take only integer values
• Example: Number of credit hours, Difference in number of credit
hours this term vs last
• Continuous random variables take real (decimal) values
• Example: Cost of books this term, Difference in cost of books
this term vs last

46
Expectation

• We are often interested in the average outcome of a random

variable.
• We call this the expected value (mean), and it is a weighted
average of the possible outcomes
k
X
µ = E(X) = xi P(X = xi )
i=1

47
Expected value of a discrete random variable

In a game of cards you win $1 if you draw a heart, $5 if you draw an

ace (including the ace of hearts), $10 if you draw the king of spades
and nothing for any other card you draw. Write the probability model
for your winnings, and calculate your expected winning.

48
Expected value of a discrete random variable

In a game of cards you win $1 if you draw a heart, $5 if you draw an

ace (including the ace of hearts), $10 if you draw the king of spades
and nothing for any other card you draw. Write the probability model
for your winnings, and calculate your expected winning.

Event X P(X) X P(X)

12 12
Heart (not ace) 1 52 52
4 20
Ace 5 52 52
1 10
King of spades 10 52 52
35
All else 0 52 0

Total E(X) = 42
52 ≈ 0.81
48
Expected value of a discrete random variable (cont.)

Below is a visual representation of the probability distribution of

winnings from this game:

0.6
0.5
0.4
0.3
0.2
0.1
0.0
0 1 2 3 4 5 6 7 8 9 10

49
Variability

We are also often interested in the variability in the values of a

random variable.
k
X
σ2 = Var(X) = (xi − E(X))2 P(X = xi )
i=1

σ = SD(X) =
p
Var(X)

50
Variability of a discrete random variable

For the previous card game example, how much would you expect
the winnings to vary from game to game?

51
Variability of a discrete random variable

For the previous card game example, how much would you expect
the winnings to vary from game to game?

X P(X) X P(X) (X − E(X))2 P(X) (X − E(X))2

1 12
52
1× 12
52
= 12
52
(1 − 0.81)2 = 0.0361 12
52
× 0.0361 = 0.0083

5 4
52
5× 4
52
= 20
52
(5 − 0.81)2 = 17.5561 4
52
× 17.5561 = 1.3505

10 1
52
10 × 1
52
= 10
52
(10 − 0.81)2 = 84.4561 1
52
× 84.0889 = 1.6242

0 35
52
0× 35
52
=0 (0 − 0.81)2 = 0.6561 35
52
× 0.6561 = 0.4416

E(X) = 0.81

51
Variability of a discrete random variable

For the previous card game example, how much would you expect
the winnings to vary from game to game?

X P(X) X P(X) (X − E(X))2 P(X) (X − E(X))2

1 12
52
1× 12
52
= 12
52
(1 − 0.81)2 = 0.0361 12
52
× 0.0361 = 0.0083

5 4
52
5× 4
52
= 20
52
(5 − 0.81)2 = 17.5561 4
52
× 17.5561 = 1.3505

10 1
52
10 × 1
52
= 10
52
(10 − 0.81)2 = 84.4561 1
52
× 84.0889 = 1.6242

0 35
52
0× 35
52
=0 (0 − 0.81)2 = 0.6561 35
52
× 0.6561 = 0.4416

E(X) = 0.81 V(X) = 3.4246

51
Variability of a discrete random variable

For the previous card game example, how much would you expect
the winnings to vary from game to game?

X P(X) X P(X) (X − E(X))2 P(X) (X − E(X))2

1 12
52
1× 12
52
= 12
52
(1 − 0.81)2 = 0.0361 12
52
× 0.0361 = 0.0083

5 4
52
5× 4
52
= 20
52
(5 − 0.81)2 = 17.5561 4
52
× 17.5561 = 1.3505

10 1
52
10 × 1
52
= 10
52
(10 − 0.81)2 = 84.4561 1
52
× 84.0889 = 1.6242

0 35
52
0× 35
52
=0 (0 − 0.81)2 = 0.6561 35
52
× 0.6561 = 0.4416

E(X) = 0.81 V(X) = 3.4246

√ 51
SD(X) = 3.4246 = 1.85
Linear combinations

• A linear combination of random variables X and Y is given by

aX + bY

where a and b are some fixed numbers.

52
Linear combinations

• A linear combination of random variables X and Y is given by

aX + bY

where a and b are some fixed numbers.

• The average value of a linear combination of random
variables is given by

E(aX + bY) = a × E(X) + b × E(Y)

52
Calculating the expectation of a linear combination

On average you take 10 minutes for each statistics homework prob-

lem and 15 minutes for each chemistry homework problem. This
week you have 5 statistics and 4 chemistry homework problems as-
signed. What is the total time you expect to spend on statistics and
chemistry homework for the week?

53
Calculating the expectation of a linear combination

On average you take 10 minutes for each statistics homework prob-

lem and 15 minutes for each chemistry homework problem. This
week you have 5 statistics and 4 chemistry homework problems as-
signed. What is the total time you expect to spend on statistics and
chemistry homework for the week?

E(S + S + S + S + S + C + C + C + C) = 5 × E(S) + 4 × E(C)

= 5 × 10 + 4 × 15
= 50 + 60
= 110 min

53
Linear combinations

• The variability of a linear combination of two independent

random variables is calculated as

V(aX + bY) = a2 × V(X) + b2 × V(Y)

54
Linear combinations

• The variability of a linear combination of two independent

random variables is calculated as

V(aX + bY) = a2 × V(X) + b2 × V(Y)

• The standard deviation of the linear combination is the square

root of the variance.

54
Linear combinations

• The variability of a linear combination of two independent

random variables is calculated as

V(aX + bY) = a2 × V(X) + b2 × V(Y)

• The standard deviation of the linear combination is the square

root of the variance.

Note: If the random variables are not independent, the variance calculation gets a
little more complicated and is beyond the scope of this course.

54
Calculating the variance of a linear combination

The standard deviation of the time you take for each statistics home-
work problem is 1.5 minutes, and it is 2 minutes for each chemistry
problem. What is the standard deviation of the time you expect to
spend on statistics and physics homework for the week if you have 5
statistics and 4 chemistry homework problems assigned? Suppose
that the time it takes to complete each problem is independent of
another.

55
Calculating the variance of a linear combination

The standard deviation of the time you take for each statistics home-
work problem is 1.5 minutes, and it is 2 minutes for each chemistry
problem. What is the standard deviation of the time you expect to
spend on statistics and physics homework for the week if you have 5
statistics and 4 chemistry homework problems assigned? Suppose
that the time it takes to complete each problem is independent of
another.

V(S + S + S + S + S + C + C + C + C) = V(S) + V(S) + V(S) + V(S) + V(S) +

= 5 × V(S) + 4 × V(C)
= 5 × 1.52 + 4 × 22
= 27.25
55
Practice

A casino game costs $5 to play. If the first card you draw is red,
then you get to draw a second card (without replacement). If the
second card is the ace of clubs, you win $500. If not, you don’t win
anything, i.e. lose your $5. What is your expected profits/losses
from playing this game? Remember: profit/loss = winnings - cost.

(a) A profit of 5¢ (c) A loss of 25¢

(b) A loss of 10¢ (d) A loss of 30¢

56
Practice

A casino game costs $5 to play. If the first card you draw is red,
then you get to draw a second card (without replacement). If the
second card is the ace of clubs, you win $500. If not, you don’t win
anything, i.e. lose your $5. What is your expected profits/losses
from playing this game? Remember: profit/loss = winnings - cost.

(a) A profit of 5¢ (c) A loss of 25¢

(b) A loss of 10¢ (d) A loss of 30¢

Event Win Profit: X P(X) X × P(X)

Red, A♣ 500 500 - 5 = 495 26
52 × 1
51 = 0.0098 495 × 0.0098 = 4.851
Other 0 0 - 5 = -5 1 − 0.0098 = 0.9902 −5 × 0.9902 = −4.951
E(X) = −0.1
56
Fair game

A fair game is defined as a game that costs as much as its

expected payout, i.e. expected profit is 0.

57
Fair game

A fair game is defined as a game that costs as much as its

expected payout, i.e. expected profit is 0.

Do you think casino games in Vegas cost more or less than their
expected payouts?

57
Fair game

A fair game is defined as a game that costs as much as its

expected payout, i.e. expected profit is 0.

Do you think casino games in Vegas cost more or less than their
expected payouts?

If those games cost less than their

expected payouts, it would mean that
the casinos would be losing money on
average, and hence they wouldn’t be
able to pay for all this:
Image by Moyan Brenn on Flickr http:// www.flickr.com/ photos/ aigle dore/ 5951714693.

57
Simplifying random variables

Random variables do not work like normal algebraic variables:

X + X , 2X

58
Simplifying random variables

Random variables do not work like normal algebraic variables:

X + X , 2X

E(X + X) = E(X) + E(X) Var(X + X) = Var(X) + Var(X) (assuming independence)

= 2E(X) = 2 Var(X)

E(2X) = 2E(X) Var(2X) = 22 Var(X)

= 4 Var(X)

58
Simplifying random variables

Random variables do not work like normal algebraic variables:

X + X , 2X

E(X + X) = E(X) + E(X) Var(X + X) = Var(X) + Var(X) (assuming independence)

= 2E(X) = 2 Var(X)

E(2X) = 2E(X) Var(2X) = 22 Var(X)

= 4 Var(X)

E(X + X) = E(2X), but Var(X + X) , Var(2X).

58
Adding or multiplying?

A company has 5 Lincoln Town Cars in its fleet. Historical data show
that annual maintenance cost for each car is on average $2,154 with
a standard deviation of $132. What is the mean and the standard
deviation of the total annual maintenance cost for this fleet?

59
Adding or multiplying?

A company has 5 Lincoln Town Cars in its fleet. Historical data show
that annual maintenance cost for each car is on average $2,154 with
a standard deviation of $132. What is the mean and the standard
deviation of the total annual maintenance cost for this fleet?

Note that we have 5 cars each with the given annual maintenance
cost (X1 + X2 + X3 + X4 + X5 ), not one car that had 5 times the
given annual maintenance cost (5X).

59
Adding or multiplying?

A company has 5 Lincoln Town Cars in its fleet. Historical data show
that annual maintenance cost for each car is on average $2,154 with
a standard deviation of $132. What is the mean and the standard
deviation of the total annual maintenance cost for this fleet?

Note that we have 5 cars each with the given annual maintenance
cost (X1 + X2 + X3 + X4 + X5 ), not one car that had 5 times the
given annual maintenance cost (5X).

E(X1 + X2 + X3 + X4 + X5 ) = E(X1 ) + E(X2 ) + E(X3 ) + E(X4 ) + E(X5 )

59
Adding or multiplying?

A company has 5 Lincoln Town Cars in its fleet. Historical data show
that annual maintenance cost for each car is on average $2,154 with
a standard deviation of $132. What is the mean and the standard
deviation of the total annual maintenance cost for this fleet?

Note that we have 5 cars each with the given annual maintenance
cost (X1 + X2 + X3 + X4 + X5 ), not one car that had 5 times the
given annual maintenance cost (5X).

E(X1 + X2 + X3 + X4 + X5 ) = E(X1 ) + E(X2 ) + E(X3 ) + E(X4 ) + E(X5 )

= 5 × E(X) = 5 × 2, 154 = $10, 770

59
Adding or multiplying?

A company has 5 Lincoln Town Cars in its fleet. Historical data show
that annual maintenance cost for each car is on average $2,154 with
a standard deviation of $132. What is the mean and the standard
deviation of the total annual maintenance cost for this fleet?

Note that we have 5 cars each with the given annual maintenance
cost (X1 + X2 + X3 + X4 + X5 ), not one car that had 5 times the
given annual maintenance cost (5X).

E(X1 + X2 + X3 + X4 + X5 ) = E(X1 ) + E(X2 ) + E(X3 ) + E(X4 ) + E(X5 )

= 5 × E(X) = 5 × 2, 154 = $10, 770
Var(X1 + X2 + X3 + X4 + X5 ) = Var(X1 ) + Var(X2 ) + Var(X3 ) + Var(X4 ) + Var(X5 )

59
Adding or multiplying?

A company has 5 Lincoln Town Cars in its fleet. Historical data show
that annual maintenance cost for each car is on average $2,154 with
a standard deviation of $132. What is the mean and the standard
deviation of the total annual maintenance cost for this fleet?

Note that we have 5 cars each with the given annual maintenance
cost (X1 + X2 + X3 + X4 + X5 ), not one car that had 5 times the
given annual maintenance cost (5X).

E(X1 + X2 + X3 + X4 + X5 ) = E(X1 ) + E(X2 ) + E(X3 ) + E(X4 ) + E(X5 )

= 5 × E(X) = 5 × 2, 154 = $10, 770
Var(X1 + X2 + X3 + X4 + X5 ) = Var(X1 ) + Var(X2 ) + Var(X3 ) + Var(X4 ) + Var(X5 )
= 5 × V(X) = 5 × 1322 = $87, 120

59
Adding or multiplying?

A company has 5 Lincoln Town Cars in its fleet. Historical data show
that annual maintenance cost for each car is on average $2,154 with
a standard deviation of $132. What is the mean and the standard
deviation of the total annual maintenance cost for this fleet?

Note that we have 5 cars each with the given annual maintenance
cost (X1 + X2 + X3 + X4 + X5 ), not one car that had 5 times the
given annual maintenance cost (5X).

E(X1 + X2 + X3 + X4 + X5 ) = E(X1 ) + E(X2 ) + E(X3 ) + E(X4 ) + E(X5 )

= 5 × E(X) = 5 × 2, 154 = $10, 770
Var(X1 + X2 + X3 + X4 + X5 ) = Var(X1 ) + Var(X2 ) + Var(X3 ) + Var(X4 ) + Var(X5 )
= 5 × V(X) = 5 × 1322 = $87, 120
p
SD(X1 + X2 + X3 + X4 + X5 ) = 87, 120 = 295.16
59
Continuous distributions
Continuous distributions

• Below is a histogram of the distribution of heights of US adults.

• The proportion of data that falls in the shaded bins gives the
probability that a randomly sampled US adult is between 180
cm and 185 cm (about 5’11” to 6’1”).

140 160 180 200

height (cm)
60
From histograms to continuous distributions

Since height is a continuous numerical variable, its probability

density function is a smooth curve.

140 160 180 200

height (cm)

61
Probabilities from continuous distributions

Therefore, the probability that a randomly sampled US adult is

between 180 cm and 185 cm can also be estimated as the shaded
area under the curve.

140 160 180 200

height (cm)

62
By definition...

Since continuous probabilities are estimated as “the area under

the curve”, the probability of a person being exactly 180 cm (or any
exact value) is defined as 0.

63