AE701c Probability: Basic Concepts

Kinds of Probability

1. Classical or A Priori Probability

Consider an experiment of rolling a six-sided die. What is the probability of

the event “an even number of spots appear face up”? In the classical or a priori
approach, if an experiment can result in n mutually exclusive and equally likely
outcomes and if exactly nA of these outcomes correspond to event A, then the
probability of event A is

nA number of favorable outcomes

P ( A )= =
n total number of possible outcomes

Under the classical approach, probabilities are determined even without

performing the experiment.

2. Empirical or A Posteriori Probability

The empirical or a posteriori approach to probability is based on what is

called the law of large numbers which asserts that over a large number of trials,
the probability of an event will approach its true value. Here, the key to
establishing probabilities empirically is that more observations will provide a more
accurate estimate of the probability, that is, if an experiment has unpredictable
outcomes, and if A is one of its possible outcomes, then we say that the empirical
probability of the outcome A is the number p if, in n repeated performances of the
experiment, the long-run relative frequency ratio of occurrence of A, nA/n, tends
to the number p:

number of ×the event has occured ( n A ) n

P ( A ) = lim = lim A = p
n → ∞ number of ×the experiment is repeated ( n ) n→∞ n
Probabilities are determined based on experimental evidence under empirical
approach.

3. Subjective Probability

If there is little or no past experience or information on which to base a

probability or in the case of situations which cannot conceivably fit into the
framework of repeated outcomes under somewhat similar conditions, probability
is arrived at subjectively. In this approach, an individual assigns the likelihood of
a particular event happening based on intuition, personal beliefs, and whatever
indirect information is available.

Definitions of Terms
AE701c Probability: Basic Concepts
Probability Models
Probability models are constructed to describe events in the real world.
Equations may be used to predict the value of variables.

The two general types of probability (a priori and a posteriori) have one
important thing in common: They both require a conceptual experiment in
which the various outcomes can occur under somewhat uniform conditions.

A random (or statistical) experiment is an experiment in which:

(a) All outcomes of the experiment are known in advance.

(b) Any performance of the experiment results in an outcome that is not
known in advance.
(c) The experiment can be repeated under identical conditions.

Sample Point. Each outcome in a sample space is called an element or a

sample point.

Sample Space. The set of all possible outcomes of a random experiment is

called the sample space, and is usually denoted by Ω or S. Sample
spaces can be finite or infinite, countable or uncountable.

Event. An event is a set of outcomes of a random experiment. It is a subset of

the sample space. Events are usually denoted by the capital letters.

Simple Event and Compound Event. If an event contains only a single sample
point, it is called a simple event. A compound event is one that can be
expressed as the union of simple events.

Finite and Infinite Events/Sample Spaces. Events and sample spaces can be
finite or infinite. An event is finite if it is empty or if it consists exactly n
sample points, where n is a positive integer; otherwise it is infinite. The
same definition applies to finite and infinite sample spaces.

Countable and Uncountable Events / Sample Spaces. An event is countable

if it is finite or if its sample points or elements can be arranged in the form
of a sequence, in which case it is said to be countably infinite; otherwise
the event is uncountable. The same definition applies to countable and
uncountable sample spaces.

Set Operations Applied to Events

AE701c Probability: Basic Concepts

Subset. If every sample point of event A is also a sample point of event B, then
event A is defined to be a subset of event B, and is denoted by A  B,
which is read as “event A is contained in event B” or “event B contains
event A.”

Equal Events. Two events A and B are defined to be equal if A  B and B  A.

This is indicated as A = B.

Null or Empty Event. The null or empty event is a subset of the sample space
that contains no sample point. It is denoted by the symbol ∅ and is also
sometimes called the impossible event.

Union of Events. The union of two events A and B denoted by the symbol A 
B, is the event containing all the sample points that belong to A or to B or
to both:
A  B = { ω∨ω  A or ω  B }

The intersection of Events. The intersection of two events A and B, denoted by

A  B or AB, is the event containing all sample points that are common to
both events:

A  B = AB = { ω∨¿ ω  A , ω  B }

Difference of Events. The difference of events A and B or the relative

complement of event B with respect to event A, denoted by A – B. is the
event that consists of the sample points that are in A but are not in B:

A – B = ABc = { ω∨¿ ω  A and ω  B }

Complement of an Event. The complement of event A with respect to the

sample space Ω, denoted by A, or Ac, or Ω - A, is the event containing all
sample points that are in Ω but are not in event A:

A=Ac = { ω∨ω  Ω, ω  A}

Some Theorems on Set Operations

AE701c Probability: Basic Concepts

Thm 1 Commutative Laws. A ∪ B=B ∪ A∧ A ∩ B=B ∩ A

Thm 2 Associative Laws. A ∪ ( B ∪C ) = ( A ∪ B ) ∪C

¿ A ∩ ( B ∩C ) =( A ∩ B )∩C

Thm 3 Distributive Laws. A ∪ ( B ∩C ) = ( A ∪ B ) ∩( A ∪ C )

¿ A ∩ ( B ∪C ) = ( A ∩ B ) ∪( A ∩C )

Thm 4 ( A c )c =( A )= A

Thm 5 A Ω= A ; A ∪ Ω=Ω ; A ∩∅=∅ ;∧ A ∪∅= A

Thm 6 A A=∅ ; A ∪ A=Ω ; A ∩ A= A ;∧ A ∪ A= A

Thm 7 De Morgan’s Laws

( A ∪ B )= A ∩ B
( A ∩ B )= A ∪ B

Thm 8 A−B= A B

Disjoint and Mutually Exclusive Events. Two events A and B are mutually
exclusive events if A  B = AB = ∅. Events A and B have no sample
points in common. Events A 1 , A 2 , …are mutually exclusive if
A i A j =∅ for every i ≠ j .

Thm 9 If A and B are subsets of Ω , then

(i) A= AB ∪ A B
(ii) AB ∩ A B=∅

Thm 10 If A ⊂ B , then AB= A∧ A ∪ B=B .

Event Space A . The class of events associated with a given random experiment
is defined to be the event space. In probability theory, an event space A
is required to have the following desirable properties:

(i) Ω A
(ii) If A  A, then A  A
(iii) If A1 and A2  A , then A1A2  A

Properties (i) to (iii) are important, since if we are interested in the

occurrence of an event, then we would also be interested on event Ω, which is
the sure event. Or if we are interested in any event A, then A should also be an

AE701c Probability: Basic Concepts

event so that we can also talk about the probability that A does not occur.
Similarly, if A1 and A2 are events, so should A1A2 be events.

Probability Set Function. Let Ω be a sample space, let A be an event space,

and let P be a real-valued function defined on A. Then P is called a
probability set function or probability function, and P(A) is called the
probability of event A if the following axioms hold:

Axiom 1. (Nonnegativity) For every event A, P(A) > 0

Axiom 2. (Normalization) The probability of the entire sample space is equal to
1, that is, P(Ω) = 1
Axiom 3. (Additivity) If A and B are two mutually exclusive events in A, then the
probability of their union satisfies

P(A  B) = P(A) + P(B).

Furthermore, if A1,A2, . . . is a sequence of mutually exclusive
events in A , then the probability of their union satisfies

P(A1  A2  … ) = P(A1) + P(A2) + . . .

∞
¿ P ( ¿ i=1 ¿ ∞ A i ) =∑ P( A i )
i=1
///
Theorem 11 If ∅is the null event, then P(∅) = 0.

Theorem 12 If A1, A2, …, An are mutually exclusive events in A , then

P(A1  A2  …  An ) = P(A1) + P(A2) + . . . + P(An)
n
¿ P ( ¿ i=1 ¿ n A i ) =∑ P( A i )
i=1
///
Theorem 13 If A is the complement of event A  A , then P( A ) = 1 –
P(A).

Theorem 14 If A and B  A, then P(A) = P(AB) + P(AB)

and P(A-B) = P(A B ) = P(A) – P(AB)

Theorem 15 Additive Rule of Probability.

AE701c Probability: Basic Concepts

For any two events A and B  A , the probability that either one of A or B occurs
is
P(A  B) = P(A) + P(B) – P(AB).

More generally,
n
P ( A 1 A 2 … A n ) =∑ P ( A i ) – ∑ ∑ P ( A i A j ) + ∑ ∑ ∑ P ( A i A j A k )−(−1)n+1 P( A 1 A 2 … A n )
i=1 i< j i< j<k

Theorem 16 If A and B  A, and A  B, then P(A) < P(B)

Theorem 17 Boole’s Inequality If A1, A2, …, An  A , then

P(A1A2…An) < P(A1) + P(A2) +  + P(An)

Conditional Probability

Given a probability space (, A ,P()) and let A and B be two events in A. The
conditional probability of event B given event A, denoted by P(B|A), is defined by

P( AB )
P ( B| A ) = if P(A) > 0
P( A )

and is left undefined if P(A) = 0.

Using the notation NA to denote the number of sample points in any event A, then
for an equally likely probable sample space  with N sample points,

NA NB N AB
P( A )= , P( B )= and P( AB )=
N N N

Thus, the conditional probability formula is equivalent to

N AB
P( AB ) N N AB
P ( B| A ) = = =
P( A ) NA NA
N

AE701c Probability: Basic Concepts

Multiplication Theorem for Conditional Probability

Theorem 18 Multiplication Rule for Conditional Probability. For a

given probability space (, A ,P()), if A and B are any two
events belonging to A, for which P(AB) > 0,

then P(AB)= P(A)P(B|A) = P(B)P(A|B).

Corollary For any B1, B2, …, Bk  A,

P(B1  B2  …  Bk) = P(B1)P(B2|B1)P(B3|B1B2)P(Bk| B1  B2  … 

Bk-1)

Independence of Events

Independent and Dependent Events. For a given probability space (, A ,P()),
let A and B be two events in A . Events A and B are defined to be independent if
the fact that event A occurs does not affect the probability that event B occurs,
that is, if any one of the following conditions is satisfied:

(i) P(AB) = P(A)P(B)

(ii) P(A|B) = P(A), P(B)>0
(iii) P(B|A) = P(B), P(A)>0

Independence of Several Events

For a given probability space (, A ,P()), let A1, A2, …, Ak be k events in A .
Events A1, A2, …, Ak are defined to be independent if and only if

P(Ai, Aj) = P(Ai)P(Aj) for i  j

P(Ai AjAr) = P(Ai)P(Aj)P(Ar) for i  j  r
:
k
P ( ¿ i=1 ¿ k A i ) =∏ P( A i )
i=1

AE701c Probability: Basic Concepts