2.discrete Random Variables

2.
Discrete Random Variables
Broadband Communication Lab, NCTU, Taiwan
Outline
2.1 Basic Concepts
2.2 Probability Mass Functions
2.3 Functions of Random Variables

2.4 Expectation, Mean, and Variance
2.5 Joint PMFs of Multiple Random
Variables
2.6 Conditioning
2.7 Independence
2.1 Visualization of a Random

Variable
Main Concepts Related to Random

Variables
A random variable is a real-valued function of
the outcome of the experiment.

A function of a random variable defines
another random variable.
We can associate with each random variable
certain averages of interest, such as the
mean and the variance.
A random variable can be conditioned on an
event or on another random variables.
There is a notion of independence of a random
variable from an event or from another
random variable.
Concepts Related to Discrete

Random Variables
A discrete random variable is a real-valued
function of the outcome of the experiment

that can take a finite or countably infinite
number of values.
A discrete random variable has an associated
probability mass function (PMF), which gives
the probability of each numerical value that
the random variable can take.
A function of a discrete random variable
defines another discrete random variable,
whose PMF can be obtained from the PMF of
the original random variable.
2.2 Probability Mass Functions

X: Discrete random variable.
pX: PMF.
x: Any possible value of X.
pX (x): Probability mass of x.
(
)
Note that :
For any set S of possible values of X, we
have :
Calculation of the PMF of a Random

Variable X
For each possible value x of X:
Collect all the possible outcomes that give
rise to the event {x = X}.
Add their probabilities to obtain pX (x).
Example:
Let X be the number of heads obtained in
two independent tosses of a fair coin.
The probability of at least one head is
The Bernoulli Random Variable

Toss a coin: p(head) = p, p(tail) = 1 p.
The Bernoulli random variable X is
Its PMF is
The Binomial Random Variable

Toss a coin n times. Tosses are indep..
p(a head) = p, p(a tail) = 1 p.
X: number of heads in the n-toss sequence.
X is a binomial random variable with
parameters n and p.
PMF:
Normalization property:
The Binomial Random Variable
10
The Geometric Random Variable

Repeatedly and independently toss a
coin with p(a head) = p, 0 < p < 1.

Geometric random variable X: Number
of tosses need for a head to come up
for the first time.
PMF:
11
The Geometric Random Variable

It is a legitimate PMF because
k 1
k 1
k 1
p
(
k
)
(1
p
)
p
x
p (1 p ) k
k 0
1
p
1 (1 p)
1
12
The Poisson Random Variable

PMF:
is a positive parameter characterizing
the PMF.
13
The Poisson Random Variable

Binomial RV with small p and large n
Poisson RV with = np.
Justification: See Problem 12.
Example of Poisson RV:

Number of typos in a book.
Number of cars involved in accidents in a
city on a given day.
14
2.3 Functions of Random Variables

If Y = g(X) is a function of a RV X, then
Y is also a RV.
The PMF of Y is
15
Example 2.1 (1/2)

Example 2.1: The PMF of X is
Find the PMF of Y=|X| and Z=X2.

Sol:
16
Example 2.1 (2/2)

Possible values of y = 0, 1, 2, 3, 4.
pY(0) = pX(0) = 1/9.

pY(1) = pX(-1) + pX(1) = 2/9.
Similar for y = 2, 3, 4.
PMF of Y is
PMF of Z is
17
2.4 Expectation
We define the expected value (also
called the expectation or the mean) of a

random variable X, with PMF pX, by
Physical meaning: Center of gravity of
the PMF.
18
Example 2.2
Example 2.2 : Consider two independent coin
tosses, each with a 3/4 probability of a head,

and let X be the number of heads obtained.
This is a binomial random variable, its PMF is :
So the mean is :
19
Moments, Variance, and Standard

Deviation
nth moment = E[Xn].
The 1st moment is just the mean.
Variance: var(X) = E[(X - E[X])2].

The variance provides a measure of
dispersion of X around its mean.
Another measure of dispersion is the
standard deviation of X:
X has the same unit as X.
20
Example 2.3
Recall Example 2.1.
21
Expected Value Rule for Functions of

Random Variables
Let X be a random variable with PMF pX . Then,
the expected value of the random variable g(X)

is given by :
Proof:
22
Example 2.3 (continued)

The variance of X can also be calculated as :
23
Mean and Variance of a Linear

Function of a Random Variable
Let X be a random variable and let Y = aX + b,
where a and b are given scalars. Then,
Proof : E[Y ] (ax b) p X ( x)

x
a xp X ( x) b p X ( x)
x
aE[ x] b
24
Variance in Terms of Moments

Expression
Proof:
25
Example 2.4 (1/2)

If the weather is good (which happens with
probability 0.6), Alice walks the 2 miles to

class at a speed of V = 5 miles per hour, and
otherwise drives her motorcycle at a speed of
V = 30 miles per hour. What is the mean of
the time T to get to class?
Sol:
26
Example 2.4 (2/2)

However, it is wrong to calculate the mean of
the speed V .
To summarize, in this example we have
27
Example 2.5
Mean and Variance of the Bernoulli.
PMF:
28
Discrete Uniformly Distributed

Random Variable.
Discrete Uniformly Distributed Random
Variable.
PMF:
29
Discrete Uniformly Distributed

Random Variable.
Consider the case where a = 1 and b = n.
General case :
30
Example 2.7
The Mean of the Poisson.
PMF:
var(X) = . (See Problem 24.)

31
Example 2.8 (1/2)

Consider a quiz game where a person is given
two questions and must decide which question

to answer first. Question 1 will be answered
correctly with probability 0.8, and the person
will then receive as prize $100, while question
2 will be answered correctly with probability
0.5, and the person will then receive as prize
$200. If the first question attempted is
answered incorrectly, the quiz terminates, i.e.,
the person is not allowed to attempt the
second question. If the first question is
answered correctly, the person is allowed to
attempt the second question. Which question
should be answered first to maximize the
expected value of the total prize money
received?
32
Example 2.8 (2/2)

Sol:
33
2.5 Joint PMF of Two Random

Variables
X, Y: Two discrete random variables.
pX,Y : Joint PMF of X and Y.

(x, y): A pair of possible values of X and Y.
pX,Y (x, y) = P(X = x, Y = y).
Marginal PMFs:
Verify:
34
Example 2.9
The marginal PMF of X or Y at a given value is
obtained by adding the table entries along a

corresponding column or row.
35
Functions of Multiple Random

Variables
Z = g(X, Y) defines another random
variables.
PMF:
Expected value:
When g(X, Y) = aX + bY + c,
36
Example 2.9 (continued)

Now, a new random variable Z is defined by:
Z X 2Y
Please find E[z]=?
E[Z ] E[ X ] 2E[Y ]
3
6
8
3 51
2 3 4
20
20
20
20 20
3
7
7
3 50
E[Y ] 1 2 3 4
20
20
20
20 20
51
50
E[ Z ]
2 7.55
20
20
E[ X ] 1
Alternatively, we can find E[z] from the PFM of

variable Z. (see textbook p.95)
37
More than Two Random Variables

PMF of three random variables:
Marginal PMFs:
Expected value:
g(X1, X2,, Xn) = a1X1 + a2X2 + +anXn
38
Example 2.10
Mean of the Binomial : Your probability class
has 300 students and each student has

probability 1/3 of getting an A, independently
of any other student. What is the mean of X,
the number of students that get an A?
Sol:
1
2 1
E[ X i ] 1 0
3
3 3
39
Example 2.11
The Hat Problem : Suppose that n people
throw their hats in a box and then each picks

one hat at random. What is the expected
value of X, the number of people that get back
their own hat?
Sol: We introduce a random variable Xi that takes
the value 1 if the ith person selects his/her own

hat, and takes the value 0 otherwise.
;
40
2.6 Conditioning a Random Variable

on an Event
The conditional PMF of a random variable X,
conditioned on a particular event A with

P(A) > 0, is defined by
Note that {X = x} A are disjoint for different x,
therefore
Combining above, we can see that
so pX|A is a legitimate PMF.
41
Visualization of pX|A(x)
42
Example 2.12
Roll a Die. X: The roll of a fair six-sided die.
A: Event that the roll is an even number.

Find pX|A(x).
Sol:
43
Example 2.13
A student will take a certain test repeatedly,
up to a maximum of n times, each time with a

probability p of passing, independently of the
number of pervious attempts. What is the PMF
of the number of attempts, given that the
student passes the test?
Sol : Let A be the event that the student
passes the test.
n
P( A) (1 p) m1 p
m 1
(1 p) k 1 p
, if k 1,..., n.
n
m 1
PX | A (k ) (1 p) p
m1
0,
otherwise
44
Conditioning one Random Variable

on Another
The conditional PMF pX|Y of X given Y is
Normalization property:
The conditional PMF is often convenient for the
calculation of the joint PMF,
45
Visualization of pX|Y(x|y)
46
Example 2.14 (1/2)

Professor May B. Right often has her facts
wrong, and answers each of her students

questions incorrectly with probability 1/4,
independently of other questions. In each
lecture May is asked 0, 1, or 2 questions with
equal probability 1/3. Let X and Y be the
number of questions May is asked and the
number of questions she answers wrong in a
given lecture, respectively. Please construct
the joint PMF pX,Y (x, y).
47
Example 2.14 (2/2)

Sol:
48
Example 2.15 (1/2)

Consider a transmitter that is sending
messages over a computer network. Define

the following two random variables:
X : the travel time of a given message,
Y : the length of the given message.
Assume that the travel time X of the message

depends on its length Y. The travel time is
104Y secs with probability 1/2, 103Y secs
with probability 1/3, and 102Y secs with
probability 1/6. Please find the PMF of X.
49
Example 2.15 (2/2)

Sol: We can know:
and
So:
50
Conditional Expectations
The conditional expectation of X given
an event A with P(A) > 0, is defined by
For a function g(X), we have
The conditional expectation of X given a
value y of Y is defined by
51
Total Expectation Theorem

Total expectation theorem:
Let A1, , An form a partition of the
sample space and P(Ai) > 0 for all i, then
Let A1, , An form a partition of an
event B and
P(Ai B) > 0
for all i, then
52
Verify the Total Expectation

Theorem
Verify
53
Example 2.16
Messages Transmission :
P(Boston to New York) = 0.5.

P(Boston to Chicago) = 0.3.
P(Boston to San Francisco) = 0.2.
X: transit time of a message (random).
E[X | New York] = 0.05 s.
E[X | Chicago] = 0.1 s.
E[X | San Francisco] = 0.3 s.
Please find E[X].
Sol:
54
Example 2.17 (1/2)

Mean and Variance of the Geometric Random
Variable.
Alternative way to fine E[X] and var(X).
Define A1 = {X = 1} = {first try is a success},

A2 = {X > 1} = {first try is a failure}.
(1)
55
Example 2.17 (2/2)

So
(2)
56
2.7 Independence of a Random

Variable from an Event
We say the random variable X is
independent of the event A if
From the definition of the conditional
PMF, we have
If P(A) > 0, independence is the same
as the condition
57
Example 2.19
Consider two independent tosses of a fair coin.
Let X be the number of heads and let A be the

event that the number of heads is even. The
(unconditional) PMF of X is : ( P(A) = )
pX|A(x) = P(X = x and A)/P(A):
58
Independence of Random Variables

We say that two random variables X
and Y are independent if

or, equivalently,
X and Y are said to be conditionally
independent, given P(A) > 0, if
Once more, this is equivalent to
59
Facts About Independent Random

Variables
If X and Y are independent random
variables, then
E[XY] = E[X] E[Y].

For any functions g and h, g(X) and h(Y)
are independent.
E[g(X) h(Y)] = E[g(X)] E[h(Y)].
var(X + Y) = var(X) + var(Y).
60
Facts About Independent Random

Variables
Conditional independence may not imply
unconditional independence.
Example :
61
Independence of Several Random

Variables
Three random variables X, Y, and Z are
said to be independent if
If X, Y, and Z are independent random
variables, then
f(X), g(Y), and h(Z) are also independent.

g(X, Y) and h(Z) are independent.
g(X, Y) and h(Y, Z) are usually not
independent.
62
Variance of the Sum of Independent

Random Variables
If X1, X2, , Xn are independent random
variables, then
Example 2.20 :
Xi ~ Bernoulli(p).
X = X1 + + Xn ~ Binomial(n, p).
{Xi} are independent.
Sol:
63
Example 2.21 (1/2)

We wish to estimate the approval rating of a president,
to be called C. To this end, we ask n persons drawn at

random from the voter population, and we let Xi be a
random variable that encodes the response of the ith
person:
We model X1,X2, . . . , Xn as independent Bernoulli

random variables with common mean p and variance
p(1 p). Naturally, we view p as the true approval
rating of C. We average the responses and compute
the sample mean Sn, defined as
Please find E[Sn] and var[Sn].

64
Example 2.21 (2/2)

Sol:
65
Example 2.22 (1/2)

In many practical situations, the analytical
calculation of the probability of some event of

interest is very difficult. However, if we have a
physical or computer model that can generate
outcomes of a given experiment in accordance
with their true probabilities, we can use simulation
to calculate with high accuracy the probability of
any given event A. In particular, we independently
generate with our model n outcomes, we record
the number m that belong to the event A of
interest, and we approximate P(A) by m/n. For
example, to calculate the probability p = P(Heads)
of a biased coin, we flip the coin n times, and we
approximate p with the ratio (number of heads
recorded)/n.
66
Example 2.22 (2/2)

To see how accurate this process is, consider n
independent Bernoulli random variables X1, . . . , Xn,

each with PMF
In a simulation context, Xi corresponds to the ith

outcome, and takes the value 1 if the ith outcome
belongs to the event A. The value of the random
variable
is the estimate of P(A) provided by the simulation.

According to Example 2.21, X has mean P(A) and
variance P(A)(1P(A))/n, so that for large n, it provides
an accurate estimate of P(A).
67

2.discrete Random Variables

Uploaded by

Copyright:

Available Formats

2.discrete Random Variables

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

2.discrete Random Variables

Uploaded by

Copyright:

Available Formats

2.

Discrete Random Variables

Broadband Communication Lab, NCTU, Taiwan

2.3 Functions of Random Variables

Broadband Communication Lab, NCTU, Taiwan

2.1 Visualization of a Random

Broadband Communication Lab, NCTU, Taiwan

Main Concepts Related to Random

the outcome of the experiment.

Concepts Related to Discrete

function of the outcome of the experiment

Broadband Communication Lab, NCTU, Taiwan

2.2 Probability Mass Functions

Broadband Communication Lab, NCTU, Taiwan

Calculation of the PMF of a Random

The probability of at least one head is

Broadband Communication Lab, NCTU, Taiwan

The Bernoulli Random Variable

Broadband Communication Lab, NCTU, Taiwan

The Binomial Random Variable

Broadband Communication Lab, NCTU, Taiwan

The Binomial Random Variable

Broadband Communication Lab, NCTU, Taiwan

The Geometric Random Variable

coin with p(a head) = p, 0 < p < 1.

Broadband Communication Lab, NCTU, Taiwan

The Geometric Random Variable

Broadband Communication Lab, NCTU, Taiwan

The Poisson Random Variable

is a positive parameter characterizing

Broadband Communication Lab, NCTU, Taiwan

The Poisson Random Variable

Poisson RV with = np.

Justification: See Problem 12.

Example of Poisson RV:

Broadband Communication Lab, NCTU, Taiwan

2.3 Functions of Random Variables

Broadband Communication Lab, NCTU, Taiwan

Example 2.1 (1/2)

Find the PMF of Y=|X| and Z=X2.

Broadband Communication Lab, NCTU, Taiwan

Example 2.1 (2/2)

pY(0) = pX(0) = 1/9.

Broadband Communication Lab, NCTU, Taiwan

called the expectation or the mean) of a

Physical meaning: Center of gravity of

Broadband Communication Lab, NCTU, Taiwan

tosses, each with a 3/4 probability of a head,

Broadband Communication Lab, NCTU, Taiwan

Moments, Variance, and Standard

Variance: var(X) = E[(X - E[X])2].

X has the same unit as X.

Broadband Communication Lab, NCTU, Taiwan

Broadband Communication Lab, NCTU, Taiwan

Expected Value Rule for Functions of

the expected value of the random variable g(X)

Broadband Communication Lab, NCTU, Taiwan

Example 2.3 (continued)

Broadband Communication Lab, NCTU, Taiwan