A couple of housekeeping notes:

Notes from Friday’s recitation are posted.

Our first problem set will be available Wednesday---check
the website.
14.31/14.310 Lecture 2 We will endeavor to give you a bit of a roadmap for two
or three lectures out---please feel free to ask if you want
additional specifics.
Today: fundamentals of probability, conditional probability
and independence
Wed: Bayes’ Theorem, random variables, distributions
Next Tues: Examples of distributions, histograms

A couple of housekeeping notes: Probability

Notes from Friday’s recitation are posted. Let’s start out with some definitions
Our first problem set will be available Wednesday---check
the website. A sample space S is a collection of all possible outcomes of
an experiment.
We will endeavor to give you a bit of a roadmap for two An event A is any collection of outcomes (including
or three lectures out---please feel free to ask if you want individual outcomes, the entire sample space, the null set).
additional specifics. If the outcome is a member of an event, the event is said
Today: fundamentals of probability, conditional probability to have occurred.
and independence Wow, that’s a lot of material to get through. Event B is contained in event A is every outcome in B also
Wed: Bayes’ Theorem, random variables, distributions belongs to A.
Next Tues: Examples of distributions, histograms

Probability Probability
These are sets, so we use many of the same definitions (e.g., These are sets, so we use many of the same definitions (e.g.,
intersection, union, complement), and all results from set intersection, union, complement), and all results from set
theory apply (e.g., associative, commutative, distributive theory apply (e.g., associative, commutative, distributive
properties). Here are some other useful results: properties). Here are some other useful results:
This is how we indicate “contained in”
If A c B then AUB = B If A c B then AUB = B
If A c B and B c A then A = B If A c B and B c A then A = B
If A c B then AB = A If A c B then AB = A
AUAc = S AUAc = S
Probability Probability
These are sets, so we use many of the same definitions (e.g., These are sets, so we use many of the same definitions (e.g.,
intersection, union, complement), and all results from set intersection, union, complement), and all results from set
theory apply (e.g., associative, commutative, distributive theory apply (e.g., associative, commutative, distributive
properties). Here are some other useful results: properties). Here are some other useful results:
This is how we indicate “union”
If A c B then AUB = B If A c B then AUB = B
If A c B and B c A then A = B If A c B and B c A then A = B
If A c B then AB = A If A c B then AB = A
AUAc = S AUAc = S
This is how we indicate
“complement”

Probability Probability
These are sets, so we use many of the same definitions (e.g., Two definitions where probability theory sometimes uses
intersection, union, complement), and all results from set different terminology than set theory:
theory apply (e.g., associative, commutative, distributive
properties). Here are some other useful results: A and B are mutually exclusive (disjoint) if they have no
outcomes in common.
If A c B then AUB = B A and B are exhaustive (complementary) if their union is
If A c B and B c A then A = B S.
If A c B then AB = A
AUAc = S
This is how we indicate “intersection”
in probability

Probability---definition Probability
We will assign every event A a number P(A), which is the One can prove a lot of useful things about probabilities using
probability the event will occur (P:S-->R). set theory. We’ll just state some.
We require that
1. P(A) >= 0 for all A c S P(Ac) = 1-P(A)
2. P(S) = 1
3. For any sequence of disjoint sets A1, A2, . . ., If A c B then P(A) <= P(B)
P(UiAi) = ΣiP(Ai) For all A, 0 <= P(A) <= 1
A probability on a sample space S is a collection of numbers P(AUB) = P(A) + P(B) - P(AB)
P(A) that satisfy axioms 1-3.
P(ABc) = P(A)-P(AB)
Probability Probability
An important special case: Powerful notion: If you can put an experiment into the
framework of a simple sample space (i.e., a sample space
Suppose you have a finite sample space. Let the function where all outcomes are equally likely), all you need to do is
n(.) give the number of elements in a set. Then define count to compute probabilities of events.
P(A) = n(A)/n(S). This is called a simple sample space,
and it is a probability. What’s the probability that the sum of the faces of two fair
(Check: 1. P(A) will always be non-negative because it’s a dice come up 4 when rolled?
count. 2. P(S) will equal 1, by definition. 3. P(AUB) = n(S) = 6x6 = 36 (1&1, 1&2, 1&3, 1&4, 1&5, 1&6, 2&1, . . .)
n(AUB)/n(S) = n(A)/n(S) + n(B)/n(S) = P(A) + n(A) = 3 (1&3, 2&2, 3&1)
P(B).) so P(A) = 3/36 = 1/12

Probability Probability
If the state of Massachusetts issues 6-character license plates, What if Massachusetts does not reuse a letter or digit?
using one of 26 letters and 10 digits randomly for each Now, in the sample space, there are 36 possibilities for the
character, what is the probability that I will receive an all- 1st character, 35 left for the 2nd, and so on. n(S) =
digit license plate? 36x35x34x33x32x31 = 36!/30!.
n(S) = 36 possibilities for each of 6 characters = 366 = Similarly, in the event, there are 10 possibilities for the 1st
2.176b character, 9 left for the 2nd, and so on. n(A) =
n(A) = 10 possibilities for each of 6 characters = 106 = 1m 10x9x8x7x6x5 = 10!/4!
so P(A) = .0005 so P(A) = .0001

This is called sampling with replacement. This is called sampling without replacement.

Probability Probability
What if Massachusetts does not reuse a letter or digit? To compute these probabilities, all I did was count. Some
Now, in the sample space, there are 36 possibilities for the fancy counting, to be sure, but just counting. Here are
1st character, 35 left for the 2nd, and so on. n(S) = some rules for fancy counting (combinatorics):
36x35x34x33x32x31 = 36!/30!.
Similarly, in the event, there are 10 possibilities for the 1st
1. If an experiment has two parts, first one having m
character, 9 left for the 2nd, and so on. n(A) =
10x9x8x7x6x5 = 10!/4! possibilities and, regardless of the outcome in the first
We write n! for
part, the second one having n possibilities, then the
so P(A) = .0001 nx(n-1)x. . . 3x2x1
experiment has mxn possible outcomes.
This is called sampling without replacement.
Probability Probability---examples
2.  Any ordered arrangement of objects is called a All candidates for the Republican
permutation. The number of different permutations of Presidential nomination gather
N objects is N!. The number of different permutations onstage for an event.
of n objects taken from N objects is N!/(N-n)!.
How many handshakes are
3.  Any unordered arrangement of objects is called a
combination. The number of different combinations of n exchanged if everyone shakes
objects taken from N objects is N!/{(N-n)!n!}. We everyone else’s hand?
typically denote this ---”N choose n.”

Probability---examples Probability---examples
All candidates for the Republican All candidates for the Republican
Presidential nomination gather Presidential nomination gather
onstage for an event. onstage for an event.
How many handshakes are How many handshakes are
exchanged if everyone shakes exchanged if everyone shakes
everyone else’s hand? everyone else’s hand?
= 9x8/2

?!

Probability---examples Probability---examples
We have 40 faculty offices in the renovated E52. (Assume We have 40 faculty offices in the renovated E52. (Assume
they’re in a continuous line.) If 40 faculty members are they’re in a continuous line.) If 40 faculty members are
placed randomly in the offices, what is the probability that placed randomly in the offices, what is the probability that
Esther and I are next to each other? Esther and I are next to each other?

Actually, here we are

Probability---examples Probability---a pre-lunch example
We have 40 faculty offices in the renovated E52. (Assume The Area Four menu contains six vegetarian pizza toppings and
they’re in a continuous line.) If 40 faculty members are five non-vegetarian pizza toppings:
placed randomly in the offices, what is the probability that
Esther and I are next to each other?

2x39!/40!
If I write each on a piece of paper and randomly choose two, what
is the probability that I end up with a pizza that has one veg
and one non-veg topping?

Probability---a pre-lunch example Probability---a pre-lunch example

First characterize the sample space, S: In general, I could have chose n toppings and asked what is
S = {(V1,V2), (V1,V3), (V1,V4), . . . , (V1,N1), the probability that my pizza had n1 vegetarian toppings
(V1,N2), . . . } n(S) = = 55 and n2 non-vegetarian toppings. There would, then, be
(Are all outcomes equally likely? Yes.) possibilities for the veg toppings and for the non-veg
Now characterize A: toppings. In other words,
A = {(V1,N1), (V1,N2), . . . ,
(V2,N1), (V2,N2), . . . ,
(V3,N1), . . . } n(A) = 6x5 = 30
So the probability is n(A)/n(S) = 30/55

Probability---a pre-lunch example Probability---independence

In general, I could have chose n toppings and asked what is It’s going to be important for us, going forward, to be able to
the probability that my pizza had n1 vegetarian toppings talk about the relationship between probabilistic, or
and n2 non-vegetarian toppings. There would, then, be stochastic, events. The most fundamental of these
possibilities for the veg toppings and for the non-veg relationships is independence.
toppings. In other words, Events A and B are independent if P(AB) = P(A)P(B).
That definition doesn’t seem very intuitive, and most of us
probably think that we have a good, intuitive sense of what
independent events are. Just be careful, though, because
that intuition can be misleading.
We will refer back to this example as the basis
for a special distribution, the hypergeometric.
Probability---independence Probability---independence
Suppose you toss one die. Consider the event, A, that you Thm If A and B are independent, A and Bc are also
roll a number less than 5, and the event, B, that you roll independent.
an even number. Are these events independent? (How Pf P(ABc) = P(A)-P(AB) = P(A) - P(A)P(B) = P(A)(1-
could they be?---they rely on the same roll of a die.) P(B)) = P(A)P(Bc)
Yes, they are. Let’s check: P(A) = 2/3. P(B) = 1/2. For more than two events, we define independence the same
P(AB) = 1/3. (AB is rolling an even number less than 5, way---the events are independent if the probability of
i.e., 2 or 4.) and P(A)P(B) = P(AB). (The proper their intersection is equal to the product of their
intuition about independent events is that knowing one probabilities.
event occurred doesn’t give you any information about
whether the other occurred.)

Probability---example Probability---example
Steph Curry’s 3pt FG percentage
is 44%. (Assume independence
of shots.)
What is the probability that he
misses the next three shots he
takes, and then makes the three
after that?

Probability---example Probability---example
What is the probability that he What is the probability that he
misses the next three shots he misses three and makes three of
takes, and then makes the three the next six shots he takes?
after that?
P(miss)P(miss)P(miss)P(make)
P(make)P(make) = .563x.443
= .015
(same as any particular sequence
of 3 misses and 3 shots made---
order won’t matter.)
Probability---example Probability---example
What is the probability that he What is the probability that he
misses three and makes three of makes at least one shot in the
the next six shots he takes? next six he takes?
Just multiply the probability of any
one such sequence (.015) by the
number of such sequences
( = 20), and that
equals .30.

Probability---example Probability---example
What is the probability that he What is the probability that he
makes at least one shot? makes at least one shot?
Well, certainly could calculate Well, certainly could calculate
probability that he makes one, probability that he makes one,
the probability he makes two, the probability he makes two,
etc., and add those. There’s an etc., and add those. There’s an
easier way: easier way:
P(making at least one shot) = 1- P(making at least one shot) = 1-
P(not making any) = 1 - .566 P(not making any) = 1 - .566
= .969. = .969. We will refer back to this example as the basis
for a special distribution, the binomial.

Probability---conditional probability Probability---conditional probability

Recall that knowing that two events are independent means Recall that knowing that two events are independent means
that the occurrence (or nonoccurrence) of one event doesn’t that the occurrence (or nonoccurrence) of one event doesn’t
tell you anything about the other. tell you anything about the other.
But what if we have two events where the occurrence of one But what if we have two events where the occurrence of one
event actually tells us something relevant about the event actually tells us something relevant about the
probability of another event? How can we alter the probability of another event? How can we alter the
probability of the second event appropriately? probability of the second event appropriately?
The probability of A conditional on B, P(A|B), is P(AB)/ The probability of A conditional on B, P(A|B), is P(AB)/
P(B), assuming P(B) > 0. P(B), assuming P(B) > 0. Think about redefining both
the event and sample space
based on new information.
Probability---conditional probability Probability---example
What is the relationship between independence and conditional An interesting part of the American political process is the
probability? tension between winning over party faithful to get the
nomination and being able to appeal to a broader base of voters
Suppose A and B are independent and P(B) > 0. Then, in the general election.
P(A|B) = P(AB)/P(B) = P(A)P(B)/P(B) = P(A). Let’s suppose these candidates have following probabilities of
This is consistent with our intuition---B occurring tells us winning the nomination:
nothing about and probability of A, so the conditional Trump P(A1) = .4
probability equals the unconditional probability. (Note that Cruz P(A2) = .3
the implication goes both ways: P(A|B) = P(A) iff A & Rubio P(A3) = .2
B independent.) Carson P(A4) = .1

Probability---example Probability---example
Let’s suppose that, conditional on winning the nomination, Let’s suppose that, conditional on winning the nomination,
these candidates have following probabilities of winning the these candidates have following probabilities of winning the
general election: general election: Tension embodied in
Trump P(W|A1) = .25 Trump P(W|A1) = .25 fact that candidates
Cruz P(W|A2) = .2 Cruz P(W|A2) = .2 with higher probability
of winning nomination
Rubio P(W|A3) = .6 Rubio P(W|A3) = .6 might not have higher
Carson P(W|A4) = .4 Carson P(W|A4) = .4 probability of winning
general election.

Probability---example Probability---example
Let’s suppose that, conditional on winning the nomination, Let’s do a little side calculation:
these candidates have following probabilities of winning the P(W) = P(WS)
general election: = P(W(A1 U A2 U A3 U A4)) because A1-A4 are
Trump P(W|A1) = .25 mutually exclusive and exhaustive sets, a partition
Cruz P(W|A2) = .2 = P(WA1 U WA2 U WA3 U WA4)
Rubio P(W|A3) = .6 = P(WA1) + P(WA2) + P(WA3) + P(WA4)
Carson P(W|A4) = .4 = P(W|A1)P(A1) + P(W|A2)P(A2) + P(W|A3)P(A3)
How can we compute the probability of a Republican win in + P(W|A4)P(A4)
the general election, P(W)?
Probability---example Probability---Bayes’ Theorem
So, we just plug in to calculate the probability of a Republican We’ve seen P(AB) = P(B|A)P(A) = P(A|B)P(B) (provided
win in the general election. P(A) > 0 and P(B) > 0), so can write P(A|B) = P(B|
P(W) = .4x.25 + .3x.2 + .2x.6 + .1x.4 = .32 A)P(A)/P(B).
We’ve also seen P(B) = P(B|A)P(A) + P(B|Ac)P(Ac).
So, P(A|B) = P(B|A)P(A)/{P(B|A)P(A) + P(B|Ac)P(Ac)}.
(A & Ac form a partition of S.
You can do this with any partition
of S.)

Probability---example Probability---example
A pregnant woman lives in an area where the Zika virus is We can use Bayes’ Theorem to calculate the probability she
fairly rare---1 in 1000 people have it. Still, she’s actually has the virus conditional on her positive test.
concerned, so she gets tested. There is a good but not P(Z) = .001 (unconditional probability of having Zika)
perfect test for the virus---it gives a positive reading with P(Zc) = .999
probability .99 if the person has the virus and a positive
P(+|Z) = .99
reading with probability .05 if the person does not. Her
reading is positive. P(+|Zc) = .05
How concerned should she be now? P(Z|+) = P(+|Z)P(Z)/{P(+|Z)P(Z) + P(+|Zc)P(Zc)}
= .019---less than 2% probability!!
Surprising?

Probability---example
How can that be?
The unconditional (prior) probability of her having the virus
was quite low, 1/1000. We updated the probability based
on the results of an imperfect test, but since it’s much
more likely that the test was wrong (50 out of 1000
people without the virus test positive), our probability gets
updated based on the positive test, but it doesn’t get
updated that much. It goes from .0001 to .019.