5 Probability

The question of what probability really is does not have a totally satisfactory an-
swer. The mathematical approach is to regard it as a function which satisfies certain
‘axioms’.
Definition Let S be a sample space. A probability for S is a function P which assigns
to each event A ⊆ S a real number P(A) and satisfies the following axioms.
Axiom 1. For every event A ⊆ S we have P(A) ≥ 0
Axiom 2. P(S) = 1
Axiom 3. If A1 , A2 , . . . , An are events and Ai ∩ Aj = ∅ for all i 6= j then
n
X
P(A1 ∪ A2 ∪ · · · ∪ An ) = P(Ai ).
i=1

If A1 , A2 , . . . is a countably infinite sequence of events and Ai ∩ Aj = ∅ for all i 6= j

then ∞
X
P(A1 ∪ A2 ∪ . . . ) = P(Ai ).
i=1

This definition was first suggested by the Russian mathematician A.N. Kolmogorov
in 1933. We refer to Axioms 1, 2 and 3 as Kolmogorov’s Axioms for Probability.

We say that events A1 , A2 , A3 , . . . satisfying Ai ∩ Aj = ∅ for i 6= j are pairwise

disjoint or mutually exclusive.

Example Suppose S is a finite sample space. We showed in lectures that setting

P(A) = |A|
|S|
for each A ⊆ S gives a probability for S. This is the case when every
outcome in the sample space is equally likely.

Warning Do not assume that every outcome is equally likely without good reason.

Starting from the axioms we can deduce various properties. Hopefully, these will
agree with our intuition about probability. The proofs that all of these properties
hold are simple deductions from the axioms.

Proposition 5.1. If A is an event then

P(Ac ) = 1 − P(A).

1
Proof We have S = A ∪ Ac and A ∩ Ac = ∅ so, by Axiom 3, P(S) = P(A) + P(Ac ).
Since P(S) = 1 by Axiom 2, we have P(Ac ) = 1 − P(A). •

Corollary 5.2.
P(∅) = 0.

Proof We have S c = ∅. Proposition 5.1 and Axiom 2 now give P(∅) = 1−P(S) = 0. •

Proposition 5.3. If A and B are events and A ⊆ B then

P(A) ≤ P(B).

Proof Since A ⊆ B we have B = A ∪ (B \ A) and A ∩ (B \ A) = ∅. By Axiom 3,

P(B) = P(A) + P(B \ A). Since P(B \ A) ≥ 0 by Axiom 1, we have P(B) ≥ P(A). •

Corollary 5.4. If A is an event then P(A) ≤ 1.

Proof We have A ⊆ S. Proposition 5.3 and Axiom 2 now give P(A) ≤ P(S) = 1. •

Notation If x is an outcome of the experiment then x ∈ S and {x} ⊆ S. Hence {x}

is the event that the outcome of the experiment is x, and P({x}) is the probability
that this event occurs. We will usually write P(x) as shorthand for P({x}), although
technically the latter is more correct. Similarly we will often refer to P(x) as the
probability of the outcome x when we should really refer to it as the probability of
the simple event {x}.

Proposition 5.5. (a) If A = {a1 , a2 , . . . , an } is a finite event then

n
X
P(A) = P(ai ).
i=1

(b) If A = {a1 , a2 , a3 , . . . } is a countably infinite event then

∞
X
P(A) = P(ai ).
i=1

2
Proof We prove (b). (The proof of (a) is almost identical.) Let Ai = {ai } for all
integers i ≥ 1. Then A = A1 ∪ A2 ∪ A3 ∪ . . . and Ai ∩ Aj = ∅ for all i 6= j, so by
Axiom 3 ∞ ∞
X X
P(A) = P(Ai ) = P(ai ).
i=1 i=1
•

Proposition 5.5 tells us that we can calculate the probability of a finite or countably
infinite event by adding together the probabilities of the outcomes which belong to
it. In particular, if the whole sample space S is finite or countably infinite, then
the probabilities of all events are uniquely determined by the probabilities of the
outcomes. We will see an example later which shows that this statement is not true
when S is uncountable.
Proposition 5.6. For any two events A and B we have

P(A ∪ B) = P(A) + P(B) − P(A ∩ B).

Proof The events A \ B, B \ A and A ∩ B are pairwise disjoint and their union is
A ∪ B. By Axiom 3

P(A ∪ B) = P(A \ B) + P(B \ A) + P(A ∩ B) (1)

Axiom 3 also implies that P(A) = P(A \ B) + P(A ∩ B) so

P(A \ B) = P(A) − P(A ∩ B), (2)

and P(B) = P(B \ A) + P(A ∩ B) so

P(B) = P(B \ A) + P(A ∩ B). (3)

We can now substitute equations (2) and (3) into (1) to obtain

P(A ∪ B) = P(A) + P(B) − P(A ∩ B).

Proposition 5.7. For any three events A, B and C we have

P(A ∪ B ∪ C) = P(A) + P(B) + P(C) − P(A ∩ B) − P(A ∩ C) − P(B ∩ C) + P(A ∩ B ∩ C).

3
Proof We use Proposition 5.6. Let D = B ∪ C. By Proposition 5.6

P(A ∪ B ∪ C) = P(A ∪ D)
= P(A) + P(D) − P(A ∩ D)
= P(A) + P(B ∪ C) − P(A ∩ (B ∪ C)
= P(A) + P(B) + P(C) − P(B ∩ C) − P(A ∩ (B ∪ C). (4)

The distributive law, Lemma 2.1(c), gives

A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C).

We can apply Proposition 5.6 to the right hand side of this equation to deduce that

P(A ∩ (B ∪ C)) = P(A ∩ B) + P(A ∩ C) − P((A ∩ B) ∩ (A ∩ C)). (5)

Since (A ∩ B) ∩ (A ∩ C) = A ∩ B ∩ C, we can substitute equation (5) into (4) to

obtain

P(A ∪ B ∪ C) = P(A) + P(B) + P(C) − P(A ∩ B) − P(A ∩ C) − P(B ∩ C) + P(A ∩ B ∩ C).

Propositions 5.6 and 5.7 can be generalized to give a formula for calculating the
probability of the union of any number of events. This general result is called the
Principle of Inclusion and Exclusion. You should try to work out what the formula
is for four events. (Hint: there are 15 terms on the right hand side.)

6 Proofs and Elementary Logic

This is not the place for a formal or philosophical discussion of what proof is. However,
a willingness to think logically and to justify our assertions is a must for studying
mathematics at university level. In most cases the language and ideas we use are the
same as in normal writing but must be absolutely precise. In this short digression
there are a few brief remarks on how to understand and write proofs. You will learn
more about different sorts of proof by seeing examples as the module progresses.

We consider mathematical statements which can be either true or false. For ex-
ample the statement “x is an integer and x ≥ 2” is true when x = 3 and false when

4
x = 1 or x = 22/7. Statements which are always true are called tautologies. State-
ments which are always false are called contradictions. The negation of a statement p
is the statement not p which is false whenever p is true and true whenever p is false.
Working out the negation of a statement can be tricky so we give a few examples.
Make sure you understand all of these.
• If p is the statement x = 2 then the negation of p is the statement x 6= 2
• If p is the statement “x = 2 and y = 3” then the negation of p is the statement
“either x 6= 2 or y 6= 3” (where or is being used in its usual mathematical sense
to include both occurring).
• If p is the statement “either x = 2 or y = 3” then the negation of p is the
statement “ x 6= 2 and y 6= 3”.
• If p is the statement “every student at QM is hardworking” the negation of p
is “there exists at least student at QM who is not hardworking”.
• If p is the statement “for all x, y ∈ A with x 6= y we have f (x) = 6 f (y)”
then the negation of p is the statement “there exist x, y ∈ A with x =6 y and
f (x) = f (y)”.
Theorems (and also lemmas, propositions and corollaries) are examples of tautolo-
gies. Some tautologies are self evident, e.g. the statement “2 is an integer”. Other
tautologies require a reasoned argument to establish that they are always true e.g.
the statement “Z is countable” or the statement “if x and y are even integers then
x + y is an even integer”. To prove that a statement p is always true we have to give a
logical argument which starts with something that we know is true and uses a series
of deductions to show that p is always true. The first thing you should do when you
want to prove that p is always true is read the definitions of all the terms used to
state p. You have no hope of proving that p is always true if you do not understand
the terms used to state p. For example we cannot prove that the statement “if x and
y are even integers then x + y is an even integer” is always true without a precise
definition of what ‘even’ means.
Given two statements p and q we can make the compound statement
p⇒q
which we read as “p implies q” or equivalently “if p then q”.1 For example the
statement “if x and y are even integers then x + y is an even integer” is a combination
1
The ⇒ symbol is much abused. You should avoid using it when you write a proof. What you
write is more likely to make sense if you write it out in words.

5
of the two statements “x and y are even integers” and ”x + y is an even integer”. The
compound statement p ⇒ q is true if q is true in all situations when p is true. The
compound statement p ⇒ q is false if there exists a situation when p is true and q is
false.
Most theorems (and propositions, lemmas and corollaries) are statements of the
form p ⇒ q is true. In this case we refer to the statement p as the hypothesis of the
theorem and q as the conclusion of the theorem. A proof that p ⇒ q is true should
look something like the following:
Proof Suppose p is true. Then we have ......
So .........
Hence q is true. •

Where each sentence follows clearly from the previous ones. See for example the
proof of Lemma 4.1(a). (We suppose that “X is a finite set and f : X → Y is
injective”. We prove “|X| ≤ |Y |”.)
To prove that p ⇒ q is false it suffices to give one example when p is true and
q is false. We call such an example a counterexample to the statement p ⇒ q. For
example, to prove that the statement “f : N → N and f is injective” does not imply
that “f is surjective” we only need to give one counterexample i.e. one example of
function which is injective but not surjective.
Notice that p ⇒ q and q ⇒ p are different statements. It is quite possible that
one is true and the other is false. For example “x = 2” implies that “x2 = 4” but
“x2 = 4” does not imply that “x = 2” (we could equally well have x = −2).
The statement
p⇔q
means p ⇒ q and q ⇒ p. This is usually read as “p if and only if q” or “p and q are
equivalent”. Thus p ⇔ q is true means that p is true whenever q is true and q is true
whenever p is true. To prove that p ⇔ q is true we need to show that both p ⇒ q and
q ⇒ p are true. Sometimes it is possible to do both of these at once but it is often
clearer to prove them separately (see for example the proof of Theorem 4.2).
The statement p ⇒ q is equivalent to the statement (not q) ⇒ ( not p).2 So
another way to prove that the statement p ⇒ q is true is to show that the statement
(not q) ⇒ ( not p) is true. That is we show that whenever q is false, p is also false.
The statement (not q) ⇒ ( not p) is called the contrapositive of the statement p ⇒ q.

2
Think about why these statements are equivalent.