Chapter 1

Recap on Probability Calculations

1. Random experiment –Sample Space-Event:

 A random experiment E is an experiment whose outcomes are due to chance, and even
if repeated under the same conditions, it does not always yield the same results.
 The set of all outcomes of a random experiment, denoted Ω is called the sample space.
 An event is defined as any subset of Ω.
 An event that contains a single element of Ω is called elementary event.
 An event A is said to occur (realized) when the result of the experiment belongs to A.
 The sample space Ω can be:
 A finite sample space: for example, tossing a coin Ω = {H, T} .
 A countably infinite sample space: for example, the number of tosses before
getting heads with a coin Ω = {1,2, … } .
 An uncountably infinite sample space: a discrete Ω corresponds to a finite or
countably infinite Ω, and a continuous Ω corresponds to an uncountably
infinite Ω.

Example: Experiment of rolling a die.

The sample space is Ω = {1,2,3,4,5,6}
The sets {1}, {2}, {3}, {4}, {5}, {6} are elementary events of Ω.
The event corresponding to the appearance of an even number is 𝐵 = {2,4,6} ⊂ Ω.

Notations Set Vocabulary Probability Vocabulary

∅ Empty set Impossible event

Ω Sample space Certain event
{ω} Elementary subset of Ω Elementary event
A Subset of Ω Event
ω∈A ω belongs to A ω is one of the possible outcomes of A.
A⊂B A included in B A implies B.
𝐴∪B Union of A and B A or B
𝐴∩B Intersection of A and B A and B
𝐴𝑐 𝑜𝑟 𝐴̅ Complement of A in Ω Opposite event of A
𝐴∩B= ∅ A and B are disjoint A and B are incompatible or mutually
exclusive

1
2. Basic properties of Boolean operations:

 (𝐴𝑐 )𝑐 = 𝐴
 (𝐴 ∪ B)𝑐 = 𝐴𝑐 ∩ 𝐵𝑐 (𝐴 ∩ B)𝑐 = 𝐴𝑐 ∪ 𝐵𝑐
 (⋃𝑖∈𝐼 𝐴𝑖 )𝑐 = ⋂𝑖∈𝐼(𝐴𝑖 )𝑐 (⋂𝑖∈𝐼 𝐴𝑖 )𝑐 = ⋃𝑖∈𝐼(𝐴𝑖 )𝑐 (𝐼 finite or countable)
 𝐴 ∩ (𝐵 ∪ 𝐶 ) = (𝐴 ∩ 𝐵) ∪ (𝐴 ∩ 𝐶) 𝐴 ∪ (𝐵 ∩ 𝐶 ) = (𝐴 ∪ 𝐵) ∩ (𝐴 ∪ 𝐶)
𝑐
 𝐴 = {𝜔 ∈ Ω /𝜔 ∉ 𝐴}
 𝐴 − 𝐵 = {𝜔 ∈ Ω /𝜔 ∈ 𝐴 𝑒𝑡 𝜔 ∉ 𝐵}
 𝐴∆𝐵 = (𝐴 − 𝐵) ∪ (𝐵 − 𝐴)

3. Complete events system

The set (𝐴𝔦 ) 𝔦𝜖𝐼 it constitutes a complete events system if :

 ∀𝑖 𝐴𝑖 ≠ ∅
 ∀ 𝑖 ≠ 𝑗 𝐴𝑖 ∩ 𝐴𝑗 = ∅ events are pairwise mutually exclusive
 ∪𝑖𝜖Ι 𝐴𝑖 = Ω

𝐼 is finite or countable set.

4. Definition of probability
The probability of an event 𝐴 was defined as the relative frequency of that event. If we repeat
𝑛 times a random experiment and if 𝑛𝐴 is the number of times 𝐴 occurred, then :
𝑛
𝑃(𝐴) = lim ( 𝑛𝐴) is the empirical probability.
𝑛⟶∞

More generally, the probability of an event 𝐴 is also defined by:

number of favorable cases for 𝐴
𝑃 (𝐴 ) = .
number of possible cases

4.1 𝝈 Algebra (Tribu) of events 𝓐

A set 𝒜 of subsets of Ω is 𝜎 Algebra if it satisfies the following three conditions:

 Ω ∈ 𝒜.
 ∀ 𝐴 ∈ 𝒜, 𝐴𝒸 ∈ 𝒜
 ∀(𝐴𝑖 ) 𝑖𝜖Ι a sequence of events in 𝒜 , then ∪𝑖𝜖Ι 𝐴𝑖 ∈ 𝒜 (𝐼 finite or countable)

The pair (Ω, 𝒜 ) is called a probable space.

4.2 Axiomatic definition of probability:

Let Ω, be a sample space and 𝒜 a σ-algebra. We call probability on (Ω, 𝒜 ) any application
𝑃 from 𝒜 to [0,1 ] satisfying :

2
 𝑃 ( Ω) = 1
 For any countable sequence of pairwise disjoint events (𝐴𝑖 ) 𝑖𝜖Ι belonging to
𝒜 : 𝑃(∪𝑖𝜖Ι 𝛢𝑖 ) = ∑𝑖𝜖Ι 𝑃(𝛢𝑖 )

The application 𝑃: 𝒜 ⟶ [0,1 ] is called a probability measure on Ω.

The triplet (Ω, 𝒜, 𝑃) is called a probability space.

If Ω is finite, the σ-algebra 𝒜 is often equal to the set of subsets of Ω , that is 𝒫 (Ω).

4.3 Properties :
 𝑃 (∅) = 0
 ∀ 𝛢 ∈ 𝒜 𝑃 (𝛢 ) ≤ 1 .
 ∀ 𝛢 ∈ 𝒜 𝑃(𝛢̅) = 1 − 𝑃(𝛢)
 ∀ 𝛢, 𝛣 ∈ 𝒜 𝑖𝑓 𝛢 ⊆ 𝛣 then 𝑃(𝛢) ≤ 𝑃(𝛣) .
 ∀ 𝛢, 𝛣 ∈ 𝒜 𝑃 (𝛢 ∪ 𝛣) = 𝑃(𝛢) + 𝑃 (𝛣) − 𝑃(𝛢 ∩ 𝛣) .

4.4 Illustration of some probabilistic sets:

4.4.1 Probability on a finite set 𝛀 :

Let Ω = {ω1 , ω2 , … ωn }. To define a probability on Ω, it is enough to give 𝑛 real numbers

𝑝𝑖 such that:

 ∀ 𝑖 𝑝𝑖 ≥ 0.
 ∑𝑛𝑖=1 𝑝𝑖 = 1.

The 𝑝𝑖 are the probabilities of the elementary events {𝜔𝑖 } for 𝑖 = 1, … , 𝑛

The probability of any event 𝛢 is the sum of the probabilities of the elementary events that
constitute 𝛢.

In the case of equiprobability: all elementary events have the same probability.
1
The probabilities of elementary events are: 𝑝1 = 𝑝2 = ⋯ = 𝑝𝑛 = 𝑛

𝑐𝑎𝑟𝑑(Α)
The probability of any event can be expressed as 𝑃(Α) = 𝑐𝑎𝑟𝑑(Ω)

4.4.2 Probability on an infinite set 𝛀: Two possible cases

 Case where Ω is countably infinite Ω = {ω1 , … ωn , … }.

To define a probability on Ω, it is sufficient to provide a sequence 𝑝𝑖 of positive real numbers

such that ∑𝑛𝑖=1 𝑝𝑖 = 1.

The probability of any event 𝛢 can be calculated as: 𝑃 (𝛢) = ∑𝑗∈𝐴 𝑝𝑗

3
 Case where Ω is uncountably infinite, we define a probability on Ω using a measure 𝑚
defined on this set (where 𝑚 can be an area, volume, etc.).
𝑚(𝛢)
𝑃 (𝛢 ) =
𝑚(Ω)

5. Conditional probabilities: :
Definition: Let (Ω, 𝒜, 𝑃) be a probability space and 𝛣 be an event with non-zero probability.
For any event 𝛢, we call the conditional probability of 𝛢 given 𝛣(i.e given that 𝛣 has
occurred) the real number denoted 𝑃𝛣 (𝛢) (or 𝑃(𝛢 ∕ 𝛣)) defined by:

𝑃 (𝛢 ∩ 𝛣 )
𝑃𝛣 (𝛢) = 𝑃(𝛢 ∕ 𝛣) =
𝑃 (𝛣 )

𝑃(𝛢 ∕ 𝛣) satisfies all the properties of the probability:

𝑃𝛣 (Ω) = 𝑃 (Ω ∕ 𝛣) = 1 and 𝑃 (∅ ∕ 𝛣 ) = 0

𝑃𝛣 (𝐴̅) = 𝑃 (𝛢̅ ∕ 𝛣) = 1 − 𝑃(𝛢 ∕ 𝛣)

𝑃𝛣 (𝛢1 ∪ 𝛢2 ) = 𝑃 (𝛢1 ∪ 𝛢2 ∕ 𝛣) = 𝑃(𝛢1 ∕ 𝛣) + 𝑃 (𝛢2 ∕ Β) − 𝑃(𝛢1 ∩ 𝛢2 ∕ 𝛣)

6. Compound probabilities-Total probabilities, and

Bayes'formula
6.1 Compound probabilities
From the definition of conditional probabilities, we deduce :

𝑃 (𝛢 ∩ 𝛣 ) = 𝑃 (𝛢 ∕ 𝛣 )𝑃 (𝛣 ) = 𝑃 (𝛣 ∕ 𝛢 )𝑃 (𝛢 ) .

This formula can be generalized to 𝑛 events : Α1 , … , Α𝑛

𝑃(Α1 ∩ Α2 … ∩ Α𝑛 ) =

𝑃(Α1 )𝑃(Α2 ∕ Α1 )𝑃 (Α3 ∕ Α1 ∩ Α2 ) … … . 𝑃(Α𝑛 ⁄Α1 ∩ Α2 … ∩ Α𝑛−1 )

6.2 Total probability formula

Consider a complete events system {𝛣𝔦𝜖𝛪 } 𝔦𝜖𝛪 and 𝛢 any event in the same space. We can
write Ω =∪𝑖𝜖𝛪 𝛣𝑖 .

𝛢 = 𝛢 ∩ Ω = 𝛢 ∩ (∪𝑖𝜖𝛪 𝛣𝑖 ) =∪𝑖𝜖Ι (𝛢 ∩ 𝛣𝑖 )

Then 𝑃 (𝛢) = 𝑃(∪𝑖𝜖𝛪 (𝛢 ∩ 𝛣𝔦 )) = ∑𝑖𝜖𝛪 𝑃 (𝛢 ∩ 𝛣𝔦 ) = ∑𝑖𝜖𝛪 𝑃 (𝛢 ∕ 𝛣𝑖 )𝑃(𝛣𝑖 ) .

If Ω = 𝛣 ∪ 𝛣̅ then 𝑃 (𝛢) = 𝑃 (𝛢 ∕ 𝛣 )𝑃(𝛣 ) + 𝑃(𝛢 ∕ 𝛣̅)𝑃(𝛣̅)

4
6.3 Bayes' formula
Let {𝛣1 , … , 𝛣𝑛 } be a complete system of events in Ω and 𝛢 an event of non-zero probability.

𝑃(𝛢∩𝛣𝑗 ) 𝑃(𝛢⁄𝛣𝑗 )𝑃(𝛣𝑗 )

For every 𝑗 ∈ {1,2, … , 𝑛} we have 𝑃(𝐵𝑗 ⁄𝛢) = =∑
𝑃(𝛢) 𝑖𝜖Ι 𝑃(𝛢⁄𝛣𝑖 )𝑃(𝛣𝑖 )

𝑃(𝛢⁄𝛣 )𝑃(𝛣 )
In the case where our complete events system is {𝛣, 𝛣̅} : 𝑃(𝛣 ⁄𝛢) = 𝑃(𝛢⁄𝛣 )𝑃(𝛣 )+𝑃(𝛢∕𝛣̅)𝑃(𝛣̅)

7. Independence
7.1 Independence of 2 events
Let (Ω, 𝒜, 𝑃) be a probability space. Two events 𝛢 and 𝛣 are said to be independent if and
only if : 𝑃(𝛢 ∩ 𝛣 ) = 𝑃 (𝛢) × 𝑃(𝛣 )

The independence of the two events 𝛢 and 𝛣 is also characterized by the relations:

𝑃 (𝛢 ∕ 𝛣 ) = 𝑃 (𝛢 ) Or 𝑃 (𝛣 ∕ 𝛢 ) = 𝑃 (𝛣 )

If 𝛢 and 𝛣 are independent, then 𝛢 and 𝛣̅, 𝛢̅ and 𝛣 , 𝛢̅ and 𝛣̅ are also independent.

7.2 Mutual independence

Let 𝛢 1 … , 𝛢 𝑛 be events in Ω , They are said to be mutually independent if:

𝑃(∩𝑖𝜖Ι 𝛢𝑖 ) = ∏𝑛𝑖=1 𝑃(𝛢𝑖 ).

Mutually independence implies pairwise independence, but the converse is false.