4.

Random Variables

• Many random processes produce numbers.

These numbers are called random variables.

Examples
(i) The sum of two dice.
(ii) The length of time I have to wait at the bus
stop for a #2 bus.
(iii) The number of heads in 20 flips of a coin.
Definition. A random variable, X, is a
function from the sample space S to the real
numbers, i.e., X is a rule which assigns a number
X(s) for each outcome s ∈ S.
Example. For S = {(1, 1), (1, 2), . . . , (6, 6)} the
random variable X corresponding to the sum is
X(1, 1) = 2, X(1, 2) = 3, and in general
X(i, j) = i + j.
Note. A random variable is neither random nor a
variable. Formally, it is a function defined on S.

1
Defining events via random variables
• Notation: we write X = x for the event
{s ∈ S : X(s) = x}.
• This is different from the usual use of equality
for functions. Formally, X is a function X(s).
What does it usually mean to write f (s) = x?

• The notation is convenient since we can then

write P(X = x) to mean P ({s ∈ S : X(s) = x}).
• Example: If X is the sum of two dice, X = 4 is
the event {(1, 3), (2, 2), (3, 1)}, and
P(X = 4) = 3/36.

2
Remarks
• For any random quantity X of interest, we can
take S to be the set of values that X can take.
Then, X is formally the identity function,
X(s) = s. Sometimes this is helpful, sometimes
not.
Example. For the sum of two dice, we could
take S = {2, 3, . . . , 12}.
• It is important to distinguish between random
variables and the values they take. A
realization is a particular value taken by a
random variable.
• Conventionally, we use UPPER CASE for
random variables, and lower case (or numbers)
for realizations. So, {X = x} is the event that
the random variable X takes the specific value x.
Here, x is an arbitrary specific value, which does
not depend on the outcome s ∈ S.

3
Discrete Random Variables
Definition: X is discrete if its possible values
form a finite or countably infinite set.
Definition: If X is a discrete random variable,
then the function

p(x) = P(X = x)

is called the probability mass function

(p.m.f.) of X.

• If X has possible values x1 , x2 , . . ., then

p(xi ) > 0 and p(x) = 0 for all other values of x.
• The events X = xi , for i = 1, 2, . . . are disjoint
P
with union S, so i p(xi ) = 1.

4
Example. The probability mass function of a
random variable X is given by p(i) = c · λi /i!
for i = 0, 1, 2, . . ., where λ is some positive value.
Find
(i) P(X = 0)

(ii) P(X > 2)

5
Example. A baker makes 10 cakes on given day.
Let X be the number sold. The baker estimates
that X has p.m.f.
1 k
p(k) = + , k = 0, 1, . . . , 10
20 100
Is this a plausible probability model?

Pn
Hint. Recall that i=1 i = 12 n(n + 1). How do
you prove this?

6
Discrete distributions
• A discrete distribution is a probability mass
function, i.e. a set of values x1 , x2 , . . . and
p(x1 ), p(x2 ), . . . with 0 < p(xi ) ≤ 1 and
P
i p(xi ) = 1.

• We say that two random variables, X and Y ,

have the same distribution (or are equal in
distribution) if they have the same p.m.f.
• We say that two random variables are equal,
and write X = Y , if for all s in S, X(s) = Y (s).

Example. Roll two dice, one red and one blue.

Outcomes are listed as (red die, blue die), so
S = {(1, 1), (1, 2), . . . , (6, 6)}. Now let
X = value of red die and Y = value of blue die,
i.e.,
X(i, j) = i, Y (i, j) = j.

• X and Y have the same distribution, with

p.m.f. p(i) = 16 for i = 1, 2, . . . , 6, but X 6= Y .

7
The Cumulative Distribution Function
Definition: The c.d.f. of X is
F (x) = P(X ≤ x), for −∞ < x < ∞.

• We can also write FX (x) for the c.d.f. of X to

distinguish it from the c.d.f. FY (y) of Y .
• Some related quantities are: (i) FX (x);
(ii) FX (y); (iii) FX (X); (iv) Fx (Y ).
• Is (i) the same function as (ii)? Explain.

• Is (i) the same function as (iii)? Explain.

• What is the meaning, if any, of (iv)?

• Does it matter if we write the c.d.f of Y as

FY (x) or FY (y)? Discuss.

8
• The c.d.f. contains the same information (for a
discrete distribution) as the p.m.f., since
X
F (x) = p(xi )
xi ≤x

p(xi ) = F (xi ) − F (xi − δ)

where δ is sufficiently small that none of the

possible values lies in the interval [xi − δ, xi ).
• Sometimes it is more convenient to work with
the p.m.f. and sometimes with the c.d.f.

Example. Flip a fair coin until a head occurs.

Let X be the length of the sequence. Find the
p.m.f. of X, and plot it.
Solution.

9
Example Continued. Flip a fair coin until a
head occurs. Let X be the length of the sequence.
Find the c.d.f. of X, and plot it.
Solution

Notation. It is useful to define bxc to be the

largest integer less than or equal to x.

10
Properties of the c.d.f.
Let X be a discrete RV with possible values
x1 , x2 , . . . and c.d.f. F (x).

• 0 ≤ F (x) ≤ 1 . Why?

• F (x) is nondecreasing, i.e. if x ≤ y then

F (x) ≤ F (y). Why?

• limx→−∞ F (x) = 0 and limx→∞ F (x) = 1 .

Details are in Ross, Section 4.10.

11
Functions of a random variable
• Let X be a discrete RV with possible values
x1 , x2 , . . . and p.m.f. pX (x).
• Let Y = g(X) for some function g mapping real
numbers to real numbers. Then Y is the random
¡ ¢
variable such that Y (s) = g X(s) for each
s ∈ S. Equivalently, Y is the random variable
such that if X takes the value x, Y takes the
value g(x).

Example. If X is the outcome of rolling a fair

die, and g(x) = x2 , what is the p.m.f. of
Y = g(X) = X 2 ?
Solution.

12
 Expectation
Consider the following game. A fair die is rolled,
with the payoffs being...

Outcome Payoff ($) Probability

1 5 1/6
2,3,4 10 1/2
5,6 15 1/3

• How much would you pay to play this game?

• In the “long run”, if you played n times, the
total payoff would be roughly

n n n
×5 + × 10 + × 15 = 10.83 n
6 2 3

• The average payoff per play is ≈ $ 10.83. This is

called the expectation or expected value of
the payoff. It is also called the fair price of
playing the game.

13
Expectation of Discrete Random Variables
Definition. Let X be a discrete random variable
with possible values x1 , x2 , . . . and p.m.f. p(x).
The expected value of X is
P
E(X) = i xi p(xi )

• E(X) is a weighted average of the possible

values that X can take on.
• The expected value may not be a possible value.

Example. Flip a coin 3 times. Let X be the

number of heads. Find E(X).
Solution.

14
Expectation of a Function of a RV
• If X is a discrete random variable, and g is a
function taking real numbers to real numbers,
then g(X) is a discrete random variable also.
• If X has probability p(xi ) of taking value xi ,
then g(X) does not necessarily take value g(xi )
¡ ¢
with probability p g(xi ) . Why? Nevertheless,
£ ¤ P
Proposition. E g(X) = i g(xi ) p(xi )

Proof.

15
Example 1. Let X be the value of a fair die.
(i) Find E(X).

(ii) Find E(X 2 ).

Example 2: Linearity of expectation.

For any random variable X and constants a and b,
E(aX + b) = a · E(X) + b

Proof.

16
Example. There are two questions in a quiz
show. You get to choose the order to answer
them. If you try question 1 first, then you will be
allowed to go on to question 2, only if your
answer to question 1 is correct, vice versa. The
rewards for these two questions are V1 and V2 . If
the probability that you know the answers to the
two questions are p1 and p2 , then which question
should be chosen first?

17
Two intuitive properties of expectation
• The formula for expectation is the same as the
formula for the center of mass, when objects of
mass pi are put at position xi . In other words,
the expected value is the balancing point for the
graph of the probability mass function.
• The distribution of X is symmetric about
some point µ if p(µ + x) = p(µ − x) for every x.

If the distribution of X is symmetric about µ

then E(X) = µ. This is “obvious” from the
intuition that the center of a symmetric
distribution should also be its balancing point.

18
 Variance
• Expectation gives a measure of center of a
distribution. Variance is a measure of spread.

Definition. If X is a random variable with mean

µ, then the variance of X is
£ 2
¤
Var(X) = E (X − µ)

• The variance is the “expected squared deviation

from average.”
• A useful identity is
£ 2¤ ¡ ¢2
Var(X) = E X − E[X]

Proof.

19
Proposition. For any RV X and constants a, b,
Var(aX + b) = a2 Var(X)

Proof.

Note 1. Intuitively, adding a constant, b, should

change the center of a distribution but not change
its spread.
Note 2. The a2 reminds us that variance is
actually a measure of (spread)2 . This is
unintuitive.

20
Standard Deviation
• We might prefer to measure the spread of X in
the same units as X.

Definition. The standard deviation of X is

p
SD(X) = Var(X)

• A rule of thumb: Almost all the probability

mass of a distribution lies within two standard
deviations of the mean.

Example. Let X be the value of a die. Find (i)

E(X), (ii) Var(X), (iii) SD(X). Show the mean
and standard deviation on a graph of the p.m.f.
Solution.

21
Example: standardization. Let X be a
random variable with expected value µ and
standard deviation σ. Find the expected value
X −µ
and variance of Y = .
σ
Solution.

22
Bernoulli Random Variables
• The result of an experiment with two possible
outcomes (e.g. flipping a coin) can be classified
as either a success (with probability p) or a
failure (with probability 1 − p). Let X = 1 if
the experiment is a success and X = 0 if it is a
failure. Then the p.m.f. of X is p(1) = p,
p(0) = 1 − p.
• If the p.m.f. of a random variable can be
written as above, it is said to be Bernoulli with
parameter p.
• We write X ∼ Bernoulli(p).

23
Binomial Random Variables
Definition. Let X be the number of successes in
n independent experiments each of which is a
success (with probability p) and a failure (with
probability 1 − p). X is said to be a binomial
random variable with parameters (n, p). We write
X ∼ Binomial(n, p).

• If Xi is the Bernoulli random variable

corresponding to the ith trial, then
Pn
X = i=1 Xi .
• Whenever binomial random variables are used
as a chance model, look for the independent
trials with equal probability of success. A
chance model is only as good as its assumptions!

24
The p.m.f. of the binomial distribution
• We write 1 for a success, 0 for a failure, so e.g.
for n = 3, the sample space is
S = {000, 001, 010, 100, 011, 101, 110, 111}.
• The probability of any particular sequence with
k successes (so n − k failures) is
pk (1 − p)n−k

• Therefore, if X ∼ Binomial(n, p), then

µ ¶
n k
P(X = k) = p (1 − p)(n−k)
k

for k = 0, 1, . . . , n.

• Where are independence and constant success

probability used in this calculation?

25
Example. A die is rolled 12 times. Find an
expression for the chance that 6 appears 3 or
more times.
Solution.

26
The Binomial Theorem. Suppose that
Pn
X ∼ Binomial(n, p). Since k=0 P(X = k) = 1,
we get the identity
Pn ¡n¢ k n−k
k=0 k p (1 − p) =1

Example. For the special case p = 1/2 we obtain

Pn ¡n¢
k=0 k = 2n
This can also be calculated by counting subsets of
a set with n elements:

27
Expectation of the binomial distribution
Let X ∼ Binomial(n, p). What do you think the
expected value of X ought to be? Why?

Now check this by direct calculation...

28
Variance of the binomial distribution.
Let X ∼ Binomial(n, p). Show that

Var(X) = np(1 − p)
¡¢2
Solution. We know E[X] = n2 p2 . We have to
find E[X 2 ].

29
Discussion problem. A system of n satellites
works if at least k satellites are working. On a
cloudy day, each satellite works independently
with probability p1 and on a clear day with
probability p2 . If the chance of being cloudy is α,
what is the chance that the system will be
working?
Solution

30
Binomial probabilities for large n, small p.
Let X ∼ Binomial(N, p/N ). We look for a limit
as N becomes large.
(1). Write out the binomial probability. Take
limits, recalling that the limit of a product is the
product of the limits.

31
 h ¡ ¢ i
p N
(2). Note that log limN →∞ 1 − N =
h¡ ¢ i
N
limN →∞ log 1 − Np . Why?

¡ ¢
p N
(3). Hence, show that limN →∞ 1 − N = e−p .

(4). Using (3) and (1), obtain limN →∞ P(X = k).

32
The Poisson Distribution
• Binomial distributions with large n, small p
occur often in the natural world

Example: Nuclear decay. A large number of

unstable Uranium atoms decay independently,
with some probability p in a fixed time interval.
Example: Prussian officers. In the 19th
century Germany, each officer has some chance p
to be killed by a horse-kick each year.
Definition. A random variable X, taking on one
of the values 0, 1, 2, . . . is said to be a Poisson
random variable with parameter λ > 0 if
k
−λ λ
p(k) = P(X = k) = e ,
k!
for k = 0, 1, . . ..

33
Example. The probability of a product being
defective is 0.001. Compare the binomial
distribution with the Poisson approximation for
finding the probability that a sample of 1000
items contain exactly 2 defective item.
Solution

34
Discussion problem. A cosmic ray detector
counts, on average, ten events per day. Find the
chance that no more than three are recorded on a
particular day.
Solution. It may be surprising that there is
enough information in this question to provide a
reasonably unambiguous answer!

35
Expectation of the Poisson distribution
k −k
±
• Let X ∼ Poisson(λ), so P(X = k) = λ e k!.
Since X is approximately Binomial(N, λ/N ), it
would not be surprising to find that
λ
E[X] = N × N = λ.

• We can show E[X] = λ by direct computation:

36
Variance of the Poisson distribution
k −k
±
• Let X ∼ Poisson(λ), so P(X = k) = λ e k!.
• The Binomial(N, λ/N ) approximation suggests
λ
¡ λ
¢
Var(X) = limN → ∞N × N × 1 − N = λ.
• We can find E[X 2 ] by direct computation to
check this:

37
The Geometric Distribution
Definition. Independent trials (e.g. flipping a
coin) until a success occurs. Let X be the number
of trials required. We write X ∼ Geometric(p).
• P(X = k) = p (1 − p)k−1 , for k = 1, 2, . . .
• E(X) = 1/p and Var(X) = (1 − p)/p2

The Memoryless Property

Suppose X ∼ Geometric(p) and k, r > 0. Then

P(X > k + r|X > k) = P(X > r).

Why?

• This result shows that, conditional on no

successes before time k, X has forgotten the first
k failures, hence the Geometric distribution is
said to have a memoryless property.

38
Exercise. Let X ∼ Geometric(p). Derive the
expected value of X.
Solution.

39
Example 1. Suppose a fuse lasts for a number of
weeks X and X ∼ Geometric(1/52), so the
expected lifetime is E(X) = 52 weeks (≈ 1 year).
Should I replace it if it is still working after two
years?
Solution

Example 2. If I have rolled a die ten times and

see no 1, how long do I expect to wait (i.e. how
many more rolls do I have to make, on average)
before getting a 1?
Solution

40
The Negative Binomial Distribution
Definition. For a sequence of independent trials
with chance p of success, let X be the number of
trials until r successes have occurred. Then X has
the negative binomial distribution,
X ∼ NegBin(p, r), with p.m.f.
¡k−1¢
P(X = k) = r−1 pr (1 − p)k−r

for k = r, r + 1, r + 2, . . .

• E(X) = r/p and Var(X) = r(1 − p)/p2

• For r = 1, we can see that NegBin(p, 1) is the
same distribution as Geometric(p).

41
Example. One person in six is prepared to
answer a survey. Let X be the number of people
asked in order to get 20 responses. What is the
mean and SD of X?
Solution.

42
The Hypergeometric Distribution
Definition. n balls are drawn randomly without
replacement from an urn containing N balls of
which m are white and N − m black. Let X be
the number of white balls drawn. Then X has the
hypergeometric distribution,
X ∼ Hypergeometric(m, n, N ).
¡m¢¡N −m¢ ¡N ¢
• P(X = k) = k n−k / n , for k = 0, 1, . . . , m.
• E(X) = mn
N = np and Var(X) =
N −n
N −1 np(1 − p),
where p = m/N .
• Useful for analyzing sampling procedures.
• N here is not a random variable. We try to use
capital letters only for random variables, but this
convention is sometimes violated.

43
Example: Capture-recapture experiments.
An unknown number of animals, say N , inhabit a
certain region. To obtain some information about
the population size, ecologists catch a number,
say m of them, mark them and release them.
Later, n more are captured. Let X be the number
of marked animals in the second capture. What is
the most likely value of N ?
Solution.

44