Probability and Statistics

Welcome to
MATH F113 (Probability and Statistics)
Lecture Section : 2
Lectures 1-20

By P. H. Keskar (6067D)
Text book : Introduction to Probability and
Statistics by J. S. Milton and J. C. Arnold 4th
ed.
IC : Dr. Shivi Agarwal (Chamber 6067T)
• Evaluation : Midsem (35%) 6/3 11:00 to
12:30 PM (CB)
• Tutorials (20%)
• Compre. (45%) (OB and CB)3/5 AN
• 4 unannounced class tests will be taken in
tutorial classes for 20 marks each and for 15
mins duration, out of which 3 best will be
consider for final evaluation.

For all the course notices, see Department of

Mathematics notice board and NALANDA
website.
• Deterministic vs. statistical methods.
• For many experiments in practice, we
may not be able to tell precise outcome
of the experiment due to some
uncertainties, Statistics deals with such
experiments.
Statistical Methods :
• Descriptive Statistics: analysis of observed
data.
• Inferential Statistics : Using observed data
to draw conclusions about general objects.
Definitions :Population : The overall group of
objects about which conclusion is to be
drawn. vs. sample : A portion of
population obtained and used to draw
conclusions.
• Model building : Developing equations for
the purpose of prediction using observed
data.
 Mathematical Basis
Probability theory : Mathematical basis.
 Probabilities
Most common interpretation of probability
is
(A)Frequency interpretation : To decide the
probability of an event, we repeat the
experiment large number of times
independently.
p ≈ f / n n= no. of trials, f= no.
of times the event occurred in n trials,
for large n.
(B) Classical Probability : Under the
assumption that all outcomes of the
experiment are equally likely : For an
event A

no.of w ays A can occur

P(A) 
n o .o f w a ys ex p t ca n p ro ceed
( C) Personal Probability: Based on
personal judgment or feeling.

Though these are common

interpretations, to put them on firm
footing and also allow further
generalizations, we need to develop
conceptual theory of probability.
EX. 1.1.2 : Hemophilia is a sex-linked hereditary
blood defect of males characterized by delayed
clotting of blood which makes it difficult to
control bleeding even in case of minor injury.
When a woman is a carrier of classical
hemophilia, there is a 50% chance that a male
child will inherit the disease. If a carrier gives
birth to 2 sons, what is the probability that both
boys will have disease?
By which approach?
Soln.
Since son is as likely to have disease as not,
assuming the independence, all 4 outcomes
YY, YN, NY, NN are equally likely, and only 1,
YY, is favourable.
By classical prob, P[A]=1/4.
p. 17, Ex. 2 : A sample of 72 bridges in a given state is
selected, and the bridges chosen are inspected for
structural weaknesses. If 30 of the bridges sampled
have serious problems, what is the estimated
probability that the next bridge sampled in the state
will have serious structural problems? What method
of assigning probability are you using to obtain this
estimate?
 Sample spaces and events
• Definition : A sample space S of an
experiment is the set S such that each
physical outcome of that experiment
corresponds to exactly one element of
S. An element of S is called a sample
point.
• Definition : Any subset A of the sample
space S of an experiment is called an
event.
• In any sample space we have the event
 called impossible event and an event
S called certain event.

This allows us to perform various set

theoretic methods to represent sample
spaces and events.
 Examples
• Toss a coin successively 3 times and
record the upper faces (H for head and
T for tail) in sequential order.
This leads to sample space
S = {HHH, HHT, HTH, HTT, THH,
THT, TTH, TTT}
As events we can consider subsets,
e.g., empty set , S, {HHH, HHT} are
events.
Another event is that ‘head occurs
twice’ which is same as {HHT, HTH,
THH}
 More examples

• Toss a dice and record the upper face.

Here S= {1,2,3,4,5,6}
• Toss a dice and record Y if upper face
is 5 and N otherwise. Here S ={Y, N}
Example 1: The temperature is recorded
in celcius in Pilani on 1-1-2017 at 8.30
am.
Here sample space S = The set of all
reals.
Example 2 : The experiment is same as
previous example except we know that
in Pilani, temp. never exceeds 50 0C
and never goes below 0 0C.
Then S = [0, 50]
Definition : The complement of an
event A in a sample space S is an
event
A' = { x  S : x A}.
Definition : Two events C, E in a
sample space are called mutually
exclusive if EC is an empty set.
Extending to more events: Events A1,
A2,… are mutually exclusive if for
any ij
iA A 
j
Operations on events : Like sets, we can
consider union, intersection,
complements of events. They can be
conveniently demonstrated by Venn
diagrams.
 Counting

• Multiplication rule : If an experiment is

taking place in k stages and ni denote the
number of ways ith stage can occur (for
any occurrence in previous stages), then
the experiment can occur in n1 n2 …nk
ways.
• Tree diagram : Y1,1
x1 Y1,2

Y1,n2
x2

Yn1,1

xn1 Y n1, 2

Y n1 , n2
• Applications of multiplication Rule :
• Number of permutations of any r
objects from given n distinct objects
= nPr= (n!)/((n-r)!).
• Number of ways of choosing r objects
without repetition from n given
objects=
 n n!
 
r  
 
r!( n  r )!
• Ex. 1.3.20 : To get an opportunity to
enter the McNeill River Brown
(Grizzly) Bear Sanctuary in Alaska,
one must enter a lottery. For a given
year, there are 2000 individuals
entered, and of these a set of 120
names will be randomly selected.
Assume that you and a friend have
both entered into the lottery.
(a) In how many ways can a set of 120
names be randomly selected from
among the 2000 entered in the
drawing?
(b) In how many ways can the drawing
be done so that you and your friend
are both selected?
(c) What is the probability that you and
your friend will both be chosen?
Example : 10 persons in a room are wearing
badges marked 1 through 10. If 3 persons are
asked to leave simultaneously and their badge
numbers are noted, find the probability that
1) The smallest of these badge numbers is 5
2) The largest of these badge number is 5.
Note : In both cases, 5 is already there, so need
to choose remaining 2 to make the event.
EX. 1.3.21 : A firm employs 10
programmers, 8 system analysts, 4
computer engineers and 3 statisticians.
A “team” is to be chosen to handle a
new long-term project. The team will
consist of 3 programmers, 2 system
analysts, 2 computer engineers and 1
statistician.
(A)In how many ways can the team be
chosen?
(B)If the customer insists that one
particular engineer with whom he or
she has worked before be assigned to
the project, in how many ways can
the team be chosen?
Permutations of indistinguishable objects :
If we are permuting n objects of k
distinguishable types such that there are ni
(indistinguishable)objects of ith type for
i=1,2,…,k then number of
distinct arrangements of these n objects is

n!
( n1! )...( nk ! )
Ex. 1.review.32 : A computer system uses
passwords which consist of 5 letters
followed by a digit.
(A)How many passwords are possible?
(B)How many passwords consist of 3 A’s and
2 B’s and end in an even digit?
( C) If you forget a password but remember
that it has characteristics described in part
(B), what is the probability that you will
guess the password correctly on the first
attempt?
5
Ex. 29 : The configuration of a particular computer
terminal consists of a baud-rate setting, a duplex
setting and a parity setting. There are 11 possible
baud-rate settings, two parity settings (even/odd)
and two duplex settings (half/full).
a) How many configurations of the terminal are
possible?
b) In how many of them is the parity even and
duplex full?
c) A line surge occurs that causes these settings to
change at random. What is the probability that the
resulting configuration will have even parity and be
full duplex?
#30 A firm offers a choice of 10 free software
packages to buyers of their new home
computer. There are 25 packages from which to
choose. (a) In how many ways can the
selections be made? (b) Five of the packages
are computer games. How many selections are
possible if exactly three computer games are
selected? (c) Is this unusual?
  25 
Solutions : (a)  
 10 
 5  20 
(b)  
 3  7 
(c) If the probabilit y of the event in (b)  b/a
is small then the event is unusual.
Axiomatic definition of probability :

1) Let S denote the sample space of the

experiment. Then P[S]=1.
2) P[A]  0 for any event A.
3) If A1, A2, A3, … are finitely many or
infinite mutually exclusive events in S
then P[A1 A2 A3 …] = P[A1]+
P[A2]+P[A3]+ … (additivity
property).
Remark : Classical probability on a finite
sample space satisfies axioms of probability.

Clearly if for P[A] =N(A)/N(S), 0  P[A]  1

and P[S] = 1. Also if A1,A2,… are mutually
exclusive then
N( An ) = N(An) so
P[ An ] =  P[An].
• Personal probability need not always
satisfy 3rd axiom of probability. If it
does, then we say it is consistent and
take it seriously.
• If we are given that p is a probability
defined axiomatically, then we are not
required to prove axioms for p as we
assume axioms hold for p.
• If p is given by some other rule ( like
classical probability) then only after
proving the axioms for p can we say p
is a probability in axiomatic sense.
Theorem : p[] = 0.
(Note that Theorem states that the
conclusion holds for any probability
satisfying the axioms. Thus result must
be derived only by using axioms and
valid logic).
Proof : Let A =S, B = . Then AB = S. Also
A and B are mutually exclusive, since
A B = . Thus by 3rd axiom of probability,
P[AB ]= P[A] + P[B].
Hence P[S] = P[S] + P[].
Subtracting P[S] from both sides, we get the
result.
Theorem : P[A'] = 1 –P[A] for any event A.
Proof : ??
Theorem ( General addition rule) : For
any events A, B of a sample space S,
P[AB] = P[A] + P[B] – P[AB]
Idea of Proof : We use
AB = (AB)( AB')(A'B)
And 3 events on RHS are mutually
exclusive.
(Verify this)
Apply 3rd axiom to get the result.
• It follows from the above that
P[AB]  P[A] + P[B]. If A, B are
mutually exclusive then equality holds.
Generalization to 3 events : P[ABC]
= P[A] + P[B] + P[C] – P[AB]-
P[AC] –P[BC] + P[ABC].
Idea of Proof : Let D= AB.
LHS= P[DC] = P[D] + P[C]
– P[D C]
= P[AB] + P[C] – P[(AC) (BC)]
Now apply previous result to 1st and last
term.
Generalization to n events : For the
events A1, …, An of the sample space,

n n
P ( Ai )   P ( Ai )   P( A  A )i j
i 1 i 1 1 i  j  n

  P( A  A
1 i  j  k  n
i j  Ak )  ...

 (  1) n 1 P ( A1  ...  An )
Ex. 2.1.9 : Assume that in a particular military
exercise involving 2 units, Red and Blue, there
is 60% chance that Red unit will successfully
meet its objectives and a 70% chance that the
Blue unit will do so. There is 18% chance that
only Red unit will be successful. What is the
probability that both units will meet their
objectives? What is the probability that one or
the other but not both units will be successful?
Ex. 2.1.12 ( c) If A  B then P[A] 
P[B].

Note that B = (A'B)A and these 2

events are mutually exclusive.
Thus P[B] = P [A'B] + P[A]  P[A] as
the other term is nonnegative.
Exercise : If A, B are 2 events such that
P[A]=3/4 and P[B]=5/8 then show that
3/8  P[AB]  5/8.

From previous ex., P[A B]  P[B]

=5/8.
P[A']=1/4, P[B']= 3/8. Thus P[A'B'] 
1/4 + 3/8 = 5/8. By taking complement,
we get the result.
Conditional probability :
• Many a times we have to consider
events and their probabilities when
certain condition on outcomes is
already given.
• Thus whole sample space is not of
interest, only those outcomes which
satisfy given condition can then be
taken as sample space.
• This has an effect of restricting sample
space and considering the event when
the sample space is smaller. The
probability may be different as a result.
• When we are considering probability
of an event, the total sample space is
important and must be given
consideration.
Definition : Let A, B be events in a
sample space S with P[B] > 0. The
conditional probability of A given B is
defined to be :
P[A|B] = P[AB] / P[B].
Illustration : In a survey of students, 20%
of students were females and 10% of all
students were female science students.
This can be interpreted in terms of
probability thus :
Sample space S = set of all students.
Event F = set of all female students.
Event SC = set of all science students.
Given : P[F]=0.2, P[FSC]=0.1.
We can ask : What percentage of female
students are Science students?
This asks the conditional probability
P[SC|F] .
Alternatively same can be phrased as :
• What is the probability that a randomly
chosen student is science student given
the student is female?
• What is the probability that a (randomly
chosen) female student is a science
student?
• From the definition of conditional
probability follows the following
multiplication rule :
• If P[B] > 0 then
P[AB] = P[A|B]P[B].
• If P[A] >0, then we can also revert the
roles of A and B.
• If we are given any 2 of the 3
probabilities, we can determine the 3rd.
Multiplication rule for n events :Let A1, …,
An be events such that P[A1…An-1]  0.
Then
P[A1…An]=
P[A1]P[A2| A1]… P[An |A1…An-1] .
Cause-effect relationship : The
conditional probability P[A|B] can be
interpreted as the probability that event
B is a cause of event A (I.e. A is
caused by B or equivalently A is the
effect of B).
• Note that if B A then the percentage
of cases in B which come from A is
100, thus A is certainly an effect of B
(I.e. the probability that B is the cause
of A is 1).
• Thus if 30% of population lives in
slums, and 1% of population lives in
slum as well as has a TV set at home,
then the probability that a person living
in slums has a TV set is 1/30.
• This can be interpreted as : probability
that having a TV causes a person to
live in slum is 1/30.
Exercise 23 : The use of plant appearance in
Prospecting for ore deposits is called geobotanic
prospecting. One indicator of copper is a small
Mint with a mauve-colored flower. Suppose
that, for a given region, there is a 30% chance
that the soil has a high copper content and a
23% chance that the mint will be present there.
If the copper content is high, there is 70%
chance that the mint will be present.
 (A)Find the probability that the copper content
will be high and mint will be present.
(B) Find the probability that copper content
will be high given that mint is present.

Can you find the probability that copper content

is not high given mint is present? Probability
that copper content is high given mint is not
present?
 Independence
Definition: Events A1 and A2 are said to be
independent if P[A1  A2]=P[A1]P[A2].
Moreover for a finite collection
C={Ai : i=1,…,n} of events, the events are
independent if and only if for any
subcollection A(1) , …, A(m) of elements of
C,
P[A(1) … A(m)]=P[A(1)]…P[A(m)].
Notes : (1)There is difference between
two events being mutually exclusive and
independent.(Check definitions)
Give an example of mutually exclusive
events which are (I) independent, (II)
dependent. Also give example of
independent events which are (I)
mutually exclusive (II) not mutually
exclusive.
(2) For n events, they are mutually exclusive if
each pair of distinct events are so.

What about independence?

Independence  all pairs are independent.
(obvious)
Converse?
Ex. In two independent tosses of a fair coin, let
A : 1st toss is a head
B : 2nd toss is a tail
C : ocurrences of 1st and 2nd toss are same.
These events are pairwise independent but not
independent.
Example (2.3.1) : Consider the
experiment of drawing a card from a
well shuffled pack of 52 cards. Let
A1 : a spade is drawn, A2 : an honor (10,
J, Q, K,A) is drawn.
Are they independent? Why?
Note : If P[A2] 0 then A1 and A2 are
independent if and only if
P[A1|A2]=P[A1].

Thus, in the previous example, if we have an

extra information that a card is spade, this does
not influence the probability that a drawn card is
an honor.
In practice, independence occurs in the
following manner :
If an experiment consists of two
unrelated parts, their outcomes can be
thought as independent.
Examples: (1) Draw a card randomly
from a pack of cards. Put it back,
shuffle well and then pick the top card.
Record both cards in sequence. The
events : (a) first card is an ace and (b)
second card is an ace are independent.
(2) Draw a card randomly from a pack of
cards. Then without putting it back,
shuffle it well and then pick the top
card. Record the drawn cards in
sequence. Then events (a) first card is
an ace and (b) second card is an ace are
not independent.
Exercise : If A and B are independent
and 0 < P(B) < 1 then A' and B are also
independent.
Bayes’ theorem : If A1, …, An is a
collection of mutually exclusive events
whose union is the sample space S
such that P[Ai]  0 for any I and B is
any event with P[B]  0 then for any r,
P [ Ar ] P [ B | Ar ]
P [ Ar | B ]  n


i 1
P [ Ai ] P [ B | Ai ]
• Bayes’ theorem can be interpreted as
reverting the roles of cause and effect. If we
know the probabilities of certain events
causing an event (and also probabilities of
causes) then we can find probability that any
effect is the cause of the original cause.
• The assumptions on the effects A1,…,An are
essential for the validity of Bayes’ theorem.
In stead of assuming their union is S, it is
enough to check sum of their probabilities is
1. That they are mutually exclusive is still
necessary.
Fact : If A1, …, An are mutually
exclusive events and one of them must
occur (I.e., their union is the sample
space S, also called exhaustive) then
for any event B,

n
P[B] 

i 1
P [ A i ] P [ B | A i ].
Tree diagram version of the fact:

A1 P[B|A1]
B
P[A1]

P[A2] A2 P[B|A2]
B

P[An] P[B|An]
B
An
Exercise review.40 : A power network involves 3
substations A, B, C. Overload at any of these
substations might result in a blackout of the
entire network. Past history has shown that if a
substation A alone experiences overload, there is
1% chance of a network blackout. For stations B
and C these percentages are 2% and 5%
respectively. Overload at 2 or more substations
result in a blackout 5% of the time. During a heat
wave, there is a 60% chance that substation A
alone will experience overload.
For stations B and C, these percentages are 20
and 15% respectively. There is a 5% chance of an
overload at two or more stations simultaneously.
During a particular heat wave, a blackout due to
an overload occurred. Find the probability that
the overload occurred at substation A alone.