FUNDAMENTALS

OF MATHEMATICAL STATISTIS
3-6
PROBABILITY
CLASSICAL OR 'A
PRIORI)
4 . MATHEMATICAL (OR
or a trial results m " exihaustive, mutnally
experiment
Definition. If a random
outcones (or cases), out of
which are jarv01Lrable to the
exClusive equally
and likely oCcurrence (or happenlg) of
E, tIsualu
the probability p' of
OCCurrence of an ent E, then
denoted by P(E), is given by:
Nummber of fauournble cases ... (3-1)
P=PME) Total mumber of exaus
Bernoulli who was the first person to obtain a

This definition was given by James

quantitative measure of uncertainty.
from (3:1):
Remarks1. Since1 20, 1 > 0 and mSn, we get
P E)20 and P (E) Ss1 0sP (E) S1
that "the odds in favour 1 E a r e n1: (1 -

nm) or the
2.Sometimes we express (3-1) by saying
E (n- m): m.
oddsagainst are

E and is
of the event E is called the conijplementary
cvent of denoted
3 . The non-hapPpening
E non-happening of E is (1 -

m). Then the

by Eor E. The number of cases favourable to , it.,

probability q that E will not happen is given by:

q P() = " = 1 - 1 -p .(3:1a)

P(E) P(E) 1 .. (3-10

= P (E) =1 -P (E) P(E) =1- P(E) or + =

If the event E represents the happening of at least one of the events E, Ez .", E then its
complementary event E represents the happening of none of the events E, Ez .., Ey
Substituting in (31 b), we get
P(Happening of at least one of the events E, Ez, . , En)
=1- P
(None of the events E, Ez, s E, happens) .
(3:lc)
of the happening ofofantheevent
4. Probability of'p'the is also known the probability of success and
as
the probability 9' non-happening
event the as
probability of failure, i.e, (p +9=1).
5. If PE) 1, called crtain event and if P(E)
=

Eis a =
0, is called an impossible
E
event
6. We can compute the probability in (3:1)by logical reasoning, without conducting any
experiment. Since, the probability in (3-1) can be computed prior to obtaining any experimental
data, it is also termed as 'a priori' or mathematical probability.
3.4-1. Limitations of Classical Definition. This definition of classical
breaks down in the following cases: probability
(i) If the various outcomes of the random
equally probable. For example,
experiment are not equally likely o
(a) The probability that
candidate will pass in a certain test is not
a
50», since e
two possible outcomes, v1z., Success and failure
(excluding the
compartment) are not equally likely. possibility o
(b) The probability that a ceiling fan in a room will fall
is not 1/2, since the eve
of tfe fan 'falling' and not falling' though mutually
cqually likely. In fact, the probability of the fan fallingexclusive
will be
and exhaustive, are
almost zero.
(c) If a person jumps from a
running train, then ill
the
not be 50%, since in this case probability of his surviva
mutually exclusive, are not
the events survival and death, though exhaustive and
equally likely.
THEORY OF PROBABILITY-I 3-7
(i) Tf the exhaustive number of outcomes of the random experiment is infinite or
unknown.

35.STATISTICAL (OR EMPIRICA) PROBABILITY

Definition. (VON MISES). Ifan experiment is performed repeatedly under essentially
homogeneous and identical conditions, then the limiting value of the ratio of the number of
ties the event occurs to the number of trials, as the number of trials becomes indefinitely
large, is called the probability of the event, it being assumed that the linit is
of happening
finite and nique.
trials an event E happens M times, then the
Symbolically, if in N probability of the
happening of E, denoted by P(E), is given by:
M
P(E)=lim N
N N
(3-2)
Remarks 1. Since in the relative frequency approach, the probability is obtained
objectively by repetitive empirical observations, it is also known as 'Empirical Probability'.
in the of subjective probability. In
2. An experiment is unique and non-repeating only case

trials to establish the chance ot

other cases, there are a large number of experiments (r

in of empirical probability. In classical

an event. This is particularly
occurrence of so case

also, repeated experiments may be made to deduction o n the basis

verify whether a
probability be established
of certain axioms or undisputed laws is justified. Only after repeated trials it can
that the chance of head in a toss of a coin is 1/2. J. E. Kerrich conducted coin tossing
his confinement in World War lI. The
experiment with 10 sets of 1,000 tosses each during
number found by him were: 502, 511, 497, 529, 504, 476, 507, 520, 504, 529.
of heads
5,079 05079
probability of getting a head in a toss of a coin as
:
This gives the 10,000
the number of trials
Thus, the empirical probability approaches the classical probability
as

becomes indefinitely large.

is repeated a
3.5-1. Limitations of Empirical Probability. (i) If an experiment
conditions may not remain identical and
large number of times, the experimental
homogeneous.
(ii) The limit in (3:2) may not attain a unique value, however large N may be.
nuuctod at ra11douu znill coutaiu 53
 FUNDAMENTALS OF
MATHEMATIGAL STATISTIO
3-22
PROBABILITY
APPROACH TO
3-8. AXIOMATIC relates lhe theory
which closely
probability, to
The axiomatic approach of functions
and also set theory, a s
the modern metric theory in 1933. The axioma
probability with a Russian mathematician, natic
proposed A.N. Kolmogorov,
by and the statistical definitions as
'both' the classical
definition of probability includes ot them. On this basis, it is
c a s e s and o v e r c o m e s
the deficiencics of each
particular structure of the
modern theory of probability
to construct a logically perfect
possib.e enchanced requiremenls
of modern natural scicnce.
and at the s a m e lime to satisBy the relies entirely upon
of mathematical theory of probbility
The axiomatic development
the logic of deduction.
to 1933, were finally
were available prior
The diverse theorems of probability, as
It is important to remark
together into a unified axiomised system in 1933.
brought axiomatic nathematical systems, is
that probability in common with all
theory,
concerned witlh relations among undetined
solely things
define relationships betwecen
The axioms thus provide a set of rules which
the theorems can be
abstract entities. These rules can be used to deduce theorems, and
to deduce more theorems. These theorems have n o
complex
brought together
empirical meaning although they can be given an interpretation in terms of empirical

phenomenon. The important point, however, development of

is that the mathematical

probability theory is, in no way, conditional upon the interpretation given to the
theory.
More precisely, under axiomatic approach, lhe probability can be deduced from
mathematical concepts. To start with some concepts are laid down. Then sore
statements are made in respect of the properties possessed by these concepts. These
properties, often termed as "axionms" or rules of the theory, are used to frame some
theorems. These theorems are framed without any reference to the real world and are
deductions from the axioms of the theory. It does not attempt to give any physical
interpretation to the probability of an event.
3-8-1. Random Experiment, Sample Space and Elementary Events. The
theory of probability provides matheniatical models for 'real-world phenomenon'
involving games of chance such as tossing of coins and dice. We feel intuitively that
statements such as

() "The probability of gelting a "head' in one toss of an unbiased coin is "

(i) "The probability of getting a lour in a single toss of an unbiased die is ",
should hold. We also feel that the
probability of obtaining either a "5" or a "6" in a
single throw of a die,
should be the
sum of the
probabilities of a "5" and "6", viz.+ a

I n other words, probabilities should have some kind of additive

Finally, feel that in large number of
we a property
of repetitions experiment, for
exanmple, a com
tossing experiment, the
proportion of heads should be
probability should have a freqiuency mterpretation. approximalely , i.e., the
To deal with these
for the properties sensibly, we need a mathematical
probabilistic situation we have. Any such description or
node
as a random experinent, dented by E. probabilistic
E may be a coin or die
situation is relerred to

throwing experiment.
THEORY OF PROBABILITY- 3-23
Arawing of cards from a well-shufíled pack of cards, an agricultural experiment to
determine the effect for fertilizers on yield of a commodity, and so on.
Each performance in a random experiment, is called a trial. That is, all the trials
conducted under the same set of conditions The result of a
form a random expèriment.
trial in a random experiment is called an outconme, an elementary eventor a saniple pont.
The totality of all possible outcomes (i.e., sample points) of a random experiment
constitutes the saniple space.
E such that
Suppose e, e2 possible outcomes
are
of random experiment a
no two or more of them can occursimultaneously and exactly one of the outcomes e
en must occur. More specifically, with an
experimnent E, we associate a set
e ,

S= (, ez.. ey) of possible outcomes with the following properties:

() each element of S denotes a possible outcome of the experiment, and

() any trial results in an outcome that corresponds to one and only one element
of the set S.
The set S associated with an experiment E, real or conceptual, satisíying the above
two properties is called the sanple space of the experiment.
The elements or the points of a sample space associated with a random experinient
are

called elementary events of the experiment.

In general, suppose we have a random experiment with m outcomes say X1, X2
S of
m and another experiment
with n outcomes y1, Yn then the sample space
2 ,

which consists of carrying out the two experiments together has

nmn
the experiment and
outcomes and can be described by the cross product of the two sets x, X2 . , Xm

1. Yn 1. Hence in this case,

(x2 yi). (*ay2) 2 Yn).

m y1), (m y2) ( m Yn))

is usually denoted by S or and the
The sample space of a random experiment
outcomes of the random experiment are denotecd by e. ez, e3 ... or by , @3..

Def. The set of all possible outcomes of a given random experiment is called the sample
outcome or element in a sample space is
space associated with that experiment. Each possible
called a sanmple point or an elementary event.
S are denoted by n ($) and in
The number of sample points in the sample space
exhaustive number of cases associated
the classical probability theory, represent the
with the random experiment E.
universal set for all questions concerned with the
Remarks. 1. The sample space serves as

experiment. in S
space if the number of elements
2. A samplespace S is said to be finite (infinite) sample
is finite (infinite).
3. A sample space is called discrete if it
contains only finitely o r infinitely many points
while a sample space containing non
which can be
arranged into a simple sequence , o . ,

In this book, w e shall

number of is called a continuous sample space.
points
uenumerable
restrict ourselves to discrete sample spaces oniy.
 FUNDAMENTALS OF MATHEMATICAL STATISTICS
3-24
3.8-2. Event
vlhich is a disjoinl union of single eleme
Every non-enpty subset A of S,
Def.
cvent.
a random experiment E is callel an

SDSersof the saiple space of

S

follows
The notion of an event may also be defined as

"Of all the possible outcomes in the sample space of an cxperunent, sonie Outconies satishu

a specified description, which we call an event."

Remarks 1. As the empty set o is a subset of S, o is also an event, known as inposstble

event.
2. An event A, in particular, can be a single element subset of S, in which case it is known as

clementary event.
Some llustrations. We give below some illustrations to explain the concepts of
the sample space and event.
1. Consider random tossing of a single coin. The possible outcomes for this
experiment are (writing H for a "head" and T for a "tail"): H and T. Thus the sample
space 5 consists of two points ( , 2), corresponding to each possible outcome or
elementary event lsted, 1.e., 5 = {H, T| = ( 1 , (O2) and n (S) = 2, where n (S) is the total
number of sample points in S.
It
may again be remarked that no two elementary events can occur simultaneously
and exactly one of these must occur in a single trial. Thus, in the toss of a coin we
cannot obtain head and tail simultaneously and more over, we must obtain either
head or tail.
(a) In random toss of two coins (or two tosses of
2. a coin), the sample S iss
given by: S= [H, T| x {H, T| = {HH, HT, TH, TT] = {01, , , O,) and n(S) = space
4 = 2.
(b) Similarly,
in a random toss of three coins (or three tosses of a coin), the sample
space S is give by:
S =
H, T| [H, T} x {H, TI
x

=
|H, T} x {HH,
HT, TH, TTI
=
[HHH, HHT, HTH, HTT, THH, THT, TTH, TTT
=
j01, z 3, 4, Ms, g, 07, sl, (say); and
Combination of these outcomes (elementary events) form
(S) = 8 =
2 .()
the so-called events,
eg, let us consider the following events associated with
sample space in (").
E Threeheads E2 Three tails.
E Exactly one head Es Exactly two heads.
Es At least one head
E At least two heads.
Then these events are
represented by the following subsets of S:
E HHH) = (o,l;
E
E =
{TTT] =
(M%) E {0, o, l
=

=
[0h, h, asl; Es= E u Eau E =
{0, 2, , O, M_, g, 07}
E =
Ej u E4 {01, 2, 3, Osl
=

(C) In general, in a random toss of N coins, n (S) 2N =

3. (a) Let us consider a random toss of

1,2,3,,6; the sample spacesingle
f the six faces dice. Since a
we can obtain anyone
S is given by :
S =

[1,2,3, 4, 5, 6} =
[ , h, dh, , (D5, l . and (S) = 6.
THEORY OF PROBABILITY-1 3-25

random toss of two dice (or two tosses of a die), the sample space S is
(bin a

given by :
S = (1,2, 3, 4, 5, 6) x (1,2, 3, 4, 5, 6}

={(1,1), (1, 2), (1,3), (1,6),

(2, 1), (2, 2), (2, 3), (2, 6)

(6, 1, (6, 2), (6,3). (6, 6) .

(")
the first dice and 'b'
where in the point (a, b), 'a' represents the number obtained on

the number on the second dice. Obviously, (a, b) # (b, n), unless a b. =

represents
Exhaustive number of cases = n (S) = 36 = 6.
with sample space s in ("*).
Let us following events associated
consider the

E Sum of points on two dice is 5

=

6
EzSum of points on two dice is
the two dice is even
E3 Sum of points o n
E Sum of points on the two dice is odd
than 12
Sunm of the points on the two dice
is greater
Es
Sum of points on the two dice is divisible by 3
E 12
equal to two and is less
Sum is greater than or
than or equal to
E
The events E, Ez, Eg and E are represented
by the following subsets of S
4
1), (2, 3), (3, 2)) andn ($)
=

E I(1, 4), (4, =

and n (S) 5
(1, 5), (5, 1), (2, 4), (4, 2), (3, 3))
=

Ez =

obtained on each dice

dice is even if () the points
The sum points on the two
of
dice are odd.
(ii) the points obtained on each
are even or 5} x |1, 3,5
Ea 12, 4, 6) x |2,4, 6)u{1,3,
=

(4, 6), (6, 2), (6, 4), (6, 6)

(2, 2), (2, 4), (2, 6), (4, 2), (4, 4)
1), (5, 3), (5, 5)
(1, 1), (1, 3), (1, 5), (3, 1), (3, 3), (3, 5),(5,
3 x3 +3 x3 = 18.
and n (Ea) =

Similarly
Es =
12, 4, 6} x {1,3, 5) u |1, 3,5) x
|2, 4, 61.
(4, 3), (4, 5), (6, 1), (6, 3), (6, 5),
(2,1), (2, 3), (2, 5), (4, 1),
(5, 2), (5, 4), (5, 6)
4), (1, 6), (3, 2), (3, 4) (3, 6),
(1, 2), (1,
3x3+ 3 x3 = 18.
and n (E) =

36.
E =
S, n(E;) = n (S) =

Es =
ol, n(Es) =
0; 12.
if their sum is 3, 6, 9 or

the points on the two dice is divisible by 3

The sum of

1), (2, 4), (4, 2). (3, 3), (3, 6), (6, 3), (4, 5), (5, 4), (6, 6)
Eg =
{(1, 2), (2, 1), (1, 5), (5,
and n (E)= 12.
is a 'certain event'.
event' and E 7
Here Es is 'impossible
an
(S) = 6.
exhaustive number of cases = n
random toss of N dice,
(C) In general, in a
 FUNDAMENTALS OF MATHEMATICAL STATISTICS
3-26
dice tossed together. For this experiment,
the sample space
4. A coin and a are

consists of twelve points given by:

S = [H, TIx|1,2,,6)
(H, 1), (H, 2), .. (H, 6), (T, 1), (T, 2 ) , . , (T, 6)| = {01, 2 , (z, . , (12)

with the
appearance of
where , (i =1, 2, ... 6) represents a head on the coin together
a the coin together
tail on
ith number on the die and o, (i =7, 8, 12) represents
. . . ,

Obviously, n (S) 2 x6= 12.

appearance of ith number on die.
=

with the
Remark. In the above illustrations 1 to 4, if the coin and
die are unbiased, we can see
intuitively that in each of the above illustrations, the outcomes (sample points) are equally

likely to occur.
5. From an urn containing 4 balls of different colours red (R), blue (B), yellow (Y)

and white (W), draw two balls simultaneously.

The sample space for this experiment is : S = {RB, RY, RW, BY, BW, YW
where 'RB' stands for "the balls drawn are red and blue', ete. Note that RB and BR

represent the same outcome because we are drawing the balls simultaneously. Here
n (S) = C2 =6
6. In the above example, suppose that instead of drawing both the balls
simultaneously, the balls are drawn in succession with replacement, i.e., after the first
draw, the colour is noted and the ball drawn is put again in the urn before drawilg
the second ball.
The sample space now consists of 16 elementary events, and is given by:
S
R,B, Y, w}x {R, B, Y, w}
=
{RR, RB, RY, RW, BR, BB, BY, BW, YR, YB, YY, YW, WR, WB, WY, Www.
where 'RB" stands for 'the first ball is red and second blue', etc., and
16. » (S) = 4 x 4 =

7. Consider experiment
an in which two balls
drawn one by one from an urn
are
containing 2 white and 4 blue balls such that when the second ball is drawn, the first
is not replaced.
Let us number the six balls as 1, 2, 3, 4, 5 and 6; numbers 1 and
2
white ball and numbers 3, 4, 5 and 6 representing a
representing a blue bal. Suppose in a draw we
pick up balls numbered 2 and 6. Then (2, 6) is called an outcome of the
should be noted that the outcome (2, 6) is different from the experiment. It
outcome (6, 2) because i
the former case ball No. 2 is drawn first and ball No. 6
is drawn next while
case, 6th ball is drawn first and the second ball is
in the latter
drawn next.
The sample space consists of
thirty as listedpoints below:
= (1, 2 )
02 =(1,3) (1, 4) (1,5) 0g = (1, 6)
= (2, 1) 0= (2,3) 0g(2,4) y = (2,5)
10 = (2, 6)
(11=(3, 1) 12 = (3, 2) Wa = (3, 4)
16 = (4, 1) 14 (3, 5) D15 = (3, 6)
7 =
(4, 2) 18= (4, 3)
21 = (5, 1) 9 =
(4, 5) 20 = (4,6)
22 (5, 2) 23 = (5, 3)
(O4 = (5, 4)
2 = (6, 1) 27 = (6, 2) 25 =(5, 6)
2s =
(6, 3)
Thus (029=(6,4) 3o=(6, 5)
S =
0, , 0).., Oj0l and
s
n (S) =
30
=|1,2,3, 5, 61 4, x
(1, 2,3, 4,5, 61-I(1, 1), (2, 2), (3, 3),
(4, 4), (5, 5), (6,
6)
THEORY OF PHOBABILITY-1
3.27
Theevents:
E The first ball drawn is white; E2: The second ball drawn is white
E: Both the balls drawn are white; Es : Both the balls drawn are black
are represented respectively by the following sets of points
E 01 O2, 3, 4, O5, 6, W7, Mg, y, 10)
E2 1 s, 11, 12, 16, 17, 21, O2z, 26, O27
Es 1 , @gl and Es = j013, 1 4 , 1 5 , 1 8 , O19, 2 0 , 21, (25, O28, W29, 030

Remark. llustrations I to 7 given above are all examples of finite sample space, since the

sample points in each one of them are finite in number. We shall now give below illustrations
infinite sample spaces, Iaving infinite number of sample points.
of

8. Consider a random experiment of throwing a coin until head appears. The

sample space S experiment is given by:
in this

=
{H, TH, TTH, TTTH,.. . {a, , =
, 3, .
tail in the first throw, tail in the second throw and head in
wh:re TTH represents
the third throw, andl so on. Obviously n (S) is infinite.
9. Consider a random experiment of tossing a coin repeatedly until head tail or

appears twice in succession. The sample of all the possible outcomes may be
represented as:

S = {HH, TT, THH, HTT, HTHH, THTT, HTH"T, THTHH,.|;n (S) =**.
3.8-3. Acceptable Assignment of Probabiitie:. iLet ei, e2, . , ey be mutually

disjoint and exhaustive outcomes ot a random experiment so that its sample space is

S le, e2eN
to S, let us assign a real number called the
To each elententary event e, belonging Ihat:
event e,; and denoted by P(ej), Such
probabilitty of the elementary real number,
of each elementary event is a non-negative
(i) The probability
i.e., P (e)20, for i =1, 2, .., N; and
all elementary events of the
(ii) The sum of the probabilities assigned to

2 P(e) = 1.
sample space is 1, i.e.,
events of the sample space
Such an assignment of real numbers to the elementary

is called an acceptable assignment of probabilitics.

of probabilities to the elementary
Remark. There be many, in fact infinite, assignments
can
the two conditions,
s e n s e that they satisfy
which are ucceptable in the
events of a sample space of probabilities.
mentioned above, required for
an acceptable assignment from
c a n be greater
than 1. This follows immediately
event
The probability of no elementary
above. Thus, we have:0S P¢)S1, i 1,2,..,N for =

the conditions ()
and (i) given
of Probabilities
3-8-4. Natural Assignment
called natural (or equiprobable)
Det. assignment of
An probabilities is
the same probability.
event is assigined
each elementary
aSsgmient if events of the sample space S, then
two elementary
More precisely, if e; and e,
are

of probabilities,
in the case of natural assignment (3-4)
Vi+j=1,2,..N
P (e) =
P (e),
 UNDAMENTALS OF MAIHEMA TICAL STATIST

3-28 {ISTICS
using (3.4), we get
2 P (¢;)= 1, on
Since, P(S) =

i=l

P (e) i
= i=1,2,.., N (35)
of natural assignment of probabilities.
in the case
(H, T) and the
coin, s =
natur
tural
of random toss of
a
For example, in the case

of probabilities gives : P (H)

=
P(T) =

assignment
die, S 2, ...6) =
{f'i, t 2 , , Cal, (say) and the
Similarly, in a random throw of a
=|1,
natural assignment of probabilities is: P(¢) =i= 1,2,., 6.
the natural
In the problems relating throw(s) of coin(s) or die (dice),
to random
if the coins/dice are lair, uniform or
assignment of probabilities is possible only ot cards, for natural
unbiased. Similarly, in drawiny cards from a well-shulfled pack
must be made at random.
assigniment ot probabilities, all the draws
In the following discussion, whenever we use the adjective such as fair coin',
a s s u m e that the
fair unbiased
or card or a number drawn at "randonm', etc., w e will
die',
natural assignment is understood. Also if no assignment of probabilities is given in

any question, the natural assignment is to be taken.

3-8-5. Axiomatic Probability. A purely mathematical definition of probability
cannot give us the actual value of P(A), the probability of occurrence of the eventA
and this must be considered as a function defined on all events. Accordingly, as in the
case of any function we need a domain space which is the o-field B of the events,
generated by S; a range space which is the closed interval [0, 1] on the real line, and a

rule which assigns a value to every element ot the domain space B.

Definition. Probability Function. P(A) is the probabilityfunction defincd on a o-
field B of events ifthe following properties or axioms hold.
1. For ench A e B, P(A) is defined, is real and PlA) 20.
(Axioni of uon-egativity)
2. P (S) = 1 (Axiom of certainty)
3. If1A,) is any finite or infinile sequence of disjoint events in B, tlen

PA)- PA) (Axiom of additivity) (3.6)

An obvious
advantage of this definition is that it is not
concerned with assigning
numerical values to the probabilities of
events.
Remarks 1. The set function P defined on a-field B,
taking
satisfying the above three axioms is called the probability nieasure.
its values in the real line and

2. The same definition of

restrictions
probability applies to uncountable sample space except that
must be placed on S and its subsets. It is special
important to realise that fora
description probability measure,
of a three complete
d-field (-algebra) B formed from certain things
must be
specified,
subsets of S and set function P. sample space S, the viz., the
often called the
probability space. In most elementary applications, S is finiteThe lriplet (S, B, P,) is
is taken to be the colection of all subsets of S. and the
O-algebrab
3.Classical Probability as a
random experiment whose sample Specials has
Case of
Axiomatic Probability. Let us consider a
space hnite number ot
points, say N, given by :
THEORY OF PROBABILITY-I
3-31
As another illustration, let us consider the
random tussing of a con. The event that the
will stand erect on its edge, is Assigned the probability 0. coin

The study ot continuous random variable provides another illustration to the fact that
0, does not implyA 9, because in case ot continuous random variable
= P(A)
X, the probability at a
point is always zero, 1e, P (X = c) = 0 (See Chapter 5).

Theorem 3-3. Probability of tlhe complementary event A of A is given by

PA)=1 P (A). -

.(3-9)
Proof. A andA are mutually disjoint events, so that

AUAS P(AUA)= P(S)

Hence, from Axioms 2 and 3 of probability, we have

PA)+ P(A) = P(S) = 1

P(A) =1- P(A)
Cor. 1. We have P(A) =
1 -

P(A )s1 :P(A) 20, by Axiom 1]

Further, since P(A) 20(Axiom 1)
0SP(A) S$1. (3.90)

Cor. 2. P() =0, since =S and P(0) =P(S)=1-P(S) =1-1 =0.

Theorem 3.4. For any two events A and B, we have
() P(A B ) =P(B) - P (AnB) i) PAn B)= P(A)- P(A B)
Proof. From the Venn diagram, we get B AnB)u(A nB),
where A nB and A nB are disjoint events.
Hence by Axiom (3), we get

P(B) =P(AnB) +P (A B)

A B/ P(A B ) = P(B) - P (AnB) (3-10)

Gi) Similarly, we have A =(AnB)uA nB),

where ( A B ) and AnB are disjoint events. Hence, by Axiom 3, we get

P(AnB) =P(A) -

P(AnB) (311)
P(A) =P (AnB) +P(AnB)
Theorem 3-5. IfB cA, then

(i) P(An B)= P(A)-P(B),

(ii) P(B) S P(A)
A
B are AB
When B cA,B and A
n
Proof. (i)

mutually exclusive events

so that A = B u (AnB)

P(A) =P[Bu(An B))

= P(B) + P (A nB) (By Axiom 3)

P(AnB) = P(A) - P(B)

 FUNDAMENTALS OF
MATHEMATICAL STATISTICS

3-30
PROBABILITY
TERMS
GLOSSARY OF
TABLE set theory
Meaning in ternis of
Statement
( E AUB
B occurs
events A or
1. At least one of the EAB

2. Both the events A and B occur.

(0E A B
3. Neither A nor B occurs
E An B
4. EventA occurs and B does not occur.
(0EAAB

of the events A or B occurs.

5. Exactly one
n B)u(A o
B)
Not more than one of the events A or B oE (An B)u(A
6.
occurs. AcB
7. If event A occurs, so does B.
exclusiVe.
AnB =o
8. Events A and B are mutually
9. Complementary event of A. A
Universal set S.
10. Sample space
events. Find expression for the events
Example 3-23. A, B and C are
three arbitrary
noted below, in the context of A, B and C:
(ii) Both A and B, but not C, occur,
(i) Only A occurs,
(iv) At least one occurs,
(iii) All thhree events occur,
(vi) One and no more occurs,
(7) At least two oCcur,
() None occurs.
(vii) Two and no more occur,
Solution.
) (i)AnBn C, (i) AnBnC (iv) A UBuC,
An B n C ,
(o) AnBn C)u(AnB nC)u(A nBnC) u (A nBC)
nBn C)u(A n B nC)
(vi) (A n Bn C) u(A
(vii) (AnBnC) u(A nBnC)u(A n B nC)

(vii) (An Bn C) or AuBuC

39. SOME THEOREMS ON PROBABILITY

In this section, we shall prove a few simple theorems which help us evaluate the
probabilities of some complicated events in a rather simple way. In proving these
theorems, we shall follow the axiomatic approach, based on the three axioms,
discussed in § 3-8-5.
Theorem 3-2. Probability of the inpossible cvent is zero, i.e., P(o) = 0. ... (3-8)
Proof. Impossible event contains no sample point and hence the certain event S
and the impossible event Ù are mutually exclusive.

SuO=S P ( S U $) = P(S)
Hence, using Axiom 2 of probability, i.e., Axiom of
PS)+ P(0) = P(S) P(0) = 0
Additivity, we get
Remark. It may be noted P(A) =0, does not
imply that A is
practice, probability 'O is
assigned to the events which are so rare
necessarily an empty set. In
a lifetime. thatthey happen only once n
For example, if a
person who does notknow typing is asked to
manuscript of a book, the
probability of the event that he will type one page of the
mistake is 0. type it correctly without anyY
THEORY OF PROBABILITY-I
3-29
S =
le, e Cnl.
where obviously e, . . " are mutually disjoint events.

Let us now suppose that all the outcomes of the random experiment are equally likely so
that the probability of each of the singleton events le;l is 1/N, ie,

PIel=i=1,2,..
Further, let an event A consist of k distinct point f 5, say e1, ea , Then by the axiom of
additivity (Axiom 3), we get

(Using (
Number ofsample points inin A
Number of
nA) (3-7)
sample points
S
n(5)
NumberExhaustive
ofcasesfavourable to the eventA,
number of cases
which is the result obtained by the classical definition of probability.
Hence, the classical probability may be regarded as a special case of axiomatic probability.
4. It is interesting to see that there are some formal statements of the properties of events
derived from the frequency approach.
From axiomatic definition of probability, we get, P(A) = S 2 0 , as in Axiom 1.

Also, sincen(S) = N, we have on using (3.7), P (S) = S 1, as in Axiom 2.

(S)
Further in the case of two mutually exclusive (or disjoint) events A and B defined by
sample points n(A) and n(B), the number ofsample points belonging to A uBis n(A) + n().

P (A UB) =
"uB) u(A)n +(S)n(B) -

(s)
= "{A = P(A)+ P(B), as in Axiom 3.
n(S)
Hence, the statistical or frequency approach of probability may be regarded as a special
case of axiomatic probability.

3.8:6. Algebra of Events. For events A, B, C

(i) AUB= 0e S: e A or w e B) (i1) AnB = {0e S: 0E A and we B

A (A complement) {» e S :e A) (iv) A -B =
[we S: 0e A but e B)
(ii) =

() Similar generalisations for

u1 Ap A, etc.

B, (vii') B >A ^ A c B.
(vi)A cB for every o E A, ®E
s a m e elements, i.e., if A cB and B cA.
(vii) A B if and
=
only if A and B have
(ix) A and B disjoint (mutually exclusive) =AnB=¢ (null
set)
B are disjoint.
(x) A UB can be denoted by A + B if Aand
one of A and B, ie.,
(xi) A A B denotes those w belonging exactly
to

AAB ABUAB = AB +A B(disjoint events)

S, all the laws of set theory, viz., commutative laws,
Remark. Since the events are subsets of
of events.
distributive laws, De-Morgan's law, etc., hold for algebra
associative laws,
Theory of Probability 3

Exhaustive Cases (or events)

The number of all possible cases that may happen during a trial, are altogether said
to torm an exhastive casCs.

Example: ln case of throwing a die, there are six possibilities viz. fi fzssannfi.
So there are six exhaustive cases.

Favourable Cases (or events)

The cases which favour the happening of a particular event are said to be favourable
to that event.

Example: While throwing a die, the event 'multiple of 3 covers the two cases fa
and fi. So these two cases are favourable to the above event.

Probability (Mathematical/Classical)
It there be n exhaustive, uutually exclusive, equally likely cases, out of wiich m of
them are favourable to a particular event ' , then the probability of A is denoted
by P(A) and is detined as P(A)=
Cor. 1. 1The event which is bound to happen, is known as Certain event, So if
be a certain event, P(X) =1,
Cor. 2. The event which will never happen, is kuown as Impossible event. So if
A be an inpossible event, P(N)=0.
Cor. 3. Since both the unmber of favourable and exhaustive cases are always
positive, so P(A) 2 0.
Cor. 4. As unmber of favourable cases cannot exceed the mumber of exhaustive
cases, i.e., mSn, so 0S PA)S1
Cor. 5. If A denotes 1ot happeing of A. then P(4) + P(A) = 1. i.e., P(A) =
1-P(A). If P(A) = pand P(A) = q, then p +4=1

Demerits of the Classical Definition

1. The definition demands the finite value of the exhaustive cases. So if the
number of exhaustive cases are infinite, this definition fails.
2. The definition ol probability involves a term equally likely', which is almost
synonymous to the term 'eq11ally probable, So we are defining a term by
almost equivalent terms. So the definition is not logically sound.

3. There is another fallacy within the detinition, that before setting up the
definition of probability how do we know that the events are equally probable
i.e., the probability ol ther occurrence are equal

Frequency Definition (Statistical/Empirical)

In order to overcome the demerit no. 1, i.e., when the umber of exhaustive cases
are highly large or infinitely large, we nay count the mmber of frequeney of any
event A during n trials. Let fm(4) be the frequency of A within n.

So, is the relative frequency of A. So, P(A) is defined as

P'A)=lin