CENG 222

Statistical Methods for

Computer Engineering

Spring 2021-2022

Section 1
Tolga Can (Office: B-109)
e-mail: tcan@ceng.metu.edu.tr

Online Materials will be available on ODTU-Class

1
 Goals of the course

• Learn techniques and tools to be able to:

– analyze and interpret large scale data,
– apply probability theory and statistics to handle
uncertainty,
– infer facts and relationships from collected data, and
– construct simulations by sampling from arbitrary
distributions
• Acquire skills for the hot new CS field: “Data
Science”

2
 Course outline
• See the syllabus on ODTU-Class

3
 Grading

• Midterm exam – 30%

• Final exam – 30%
• 4 Assignments –32%
• 10 Quizzes (best 8 of 10) – 8 %

4
 ODTU Class

– Syllabus
– Course textbook
– Lecture slides and reading materials
– Assignments
– Grades

5
 COW
• A forum for course related discussions

6
 Textbook
• Probability and Statistics for Computer
Scientists, Second Edition, Michael Baron,
2013
• Your main resource of study for this course

7
 Probability
• Studies uncertainty
• A random experiment
– An experiment/observation which does not have a
certain outcome before it is conducted
• Examples
– Tossing a coin
– Observing the life time of a light bulb
– Number of games the Cavaliers will win this season
– Others?

8
 Sample space
• The set of all possible outcomes of a random
experiment is called the sample space
– Tossing a coin:
• Sample space = {H, T}
– Tossing two coins:
• Sample space = {HH, HT, TH, TT}
– Lifetime of a light bulb:
• Sample space = [0,+∞)

9
 Event
• Any collection of possible outcomes of an
experiment
– Any subset of the sample space
• Examples:
– Experiment: tossing two coins. Event: obtaining
exactly one head. {HT,TH}Ì{HH,HT,TH,TT}
– Experiment: lifetime of light bulb. Event: light
bulb does not last more than a month.
[0,1] Ì [0,+∞)

10
 Event
• A sample space of N possible outcomes yields
2N possible events
• Example: tossing a dice once
• Sample space = {1,2,3,4,5,6}
• Number of possible events = 26 = 64
• Example events?

11
 Notation used in the book
• Ω = sample space
• Æ = empty event
• P{E} = probability of event E

12
 Event algebra

• Union of two events: same as set union

– A È B = {x: x Î A or x Î B}
• Intersection of two events: same as set intersection
– A Ç B = {x: x Î A and x Î B}
• Complementation: same as in sets
– Ac or 𝐴̅ = {x: x Î Ω and x Ï A}
• Difference: same as in sets
– A\B = {x: x Î A and x Ï B}

13
 Disjoint and exhaustive events
• Disjoint events: If A and B have no outcomes
in common, i.e., A Ç B = Æ
– Also called mutually exclusive events
• If the union of a number of events equals the
sample space, they are called exhaustive
–AÈBÈC=Ω

14
 Complement, Union, Intersection
• 𝐴 ∪ 𝐵 = 𝐴̅ ∩ 𝐵(
• 𝐴 ∩ 𝐵 = 𝐴̅ ∪ 𝐵(
• 𝐸! ∪ 𝐸" ∪ 𝐸# ∪ 𝐸$ = 𝐸! ∩ 𝐸" ∩ 𝐸# ∩ 𝐸$
• 𝐸! ∩ 𝐸" ∩ 𝐸# ∩ 𝐸$ = 𝐸! ∪ 𝐸" ∪ 𝐸# ∪ 𝐸$

15
 Probability
• Assignment of a real number to an event
– The relative frequency of occurrence of an event in
a large number of experiments
• P(A)
• Axioms of probability:
– P(A) ³ 0
– P(Ω) = 1
– If A and B are mutually exclusive events, then
P(AÈB) = P(A)+P(B)
• Any function that satisfies these axioms is
called a probability function 16
 Example
• Experiment:
– Tossing two coins
– A = {obtaining exactly one head}
– P(A) = ?

17
 Computing probabilities
• for non-“mutually exclusive” events:
– 𝑷(𝐴 ∪ 𝐵) = 𝑷(𝐴) + 𝑷(𝐵) − 𝑷(𝐴 ∩ 𝐵)

18
 Independent Events
• 𝑷(𝐸! ∩ 𝐸" ∩ 𝐸# ) = 𝑃 𝐸! . 𝑃 𝐸" . 𝑃 𝐸#

19
 Applications in reliability
• Example 2.18
• Example 2.19
• Example 2.20

20
 Conditional probability
• Updating of the sample space based on new
information
• Consider two events A and B. Suppose that the
event B has occurred. This information will
change the probability of event A.
• P(A|B) denotes the conditional probability of
event A given that B has occurred.

21
 Conditional probability
• If A and B are events in Ω and P(B)>0, then
P(A|B) is called the conditional probability of
A given B if the following axiom is satisfied:
– P(A|B) = P(AÇB)/P(B)
• Example: tossing a fair dice.
– A = {the number on the dice is even}
– B = {the number on the dice < 4}
– P(A|B) = ?

22
 Independence
• If P(A|B)=P(A) we call that event A is
independent of event B
• Note:
– if two events A and B are independent, then
P(AÇB)=P(A)P(B)
• Show that P(B|A)=P(B) also holds in this case.
– In other words, A and B are mutually independent
• This does NOT mean that they are disjoint. If
A and B are disjoint then P(B|A)=0
23
 Independence
• Example: tossing a fair dice.
– A = {the number on the dice is even}
– B = {the number on the dice > 2}
– P(A|B) = ?
– P(B|A) = ?
– P(A) = ?
– P(B) = ?
• Example 2.31

24
 Bayes’ Rule
• Using conditional probability formula we may
write:
– P(A|B) = P(AÇB)/P(B)
– P(B|A) = P(AÇB)/P(A)
– à P(AÇB) = P(A|B)P(B) = P(B|A)P(A) à
P(B|A) = P(A|B)P(B) / P(A)
• This is known as the Bayes’ rule
• It forms the basis of Bayesian statistics
• What additional probabilities do we need to
know to solve Example 2.32?
25
 Law of Total Probability
• Let B1, B2, B3, ….,Bk be a partition of the
sample space. Bis are mutually disjoint. Let A
be any event.
• Note that Bis also partition A
• Then for each i = 1,2,…,k
P( A | Bi ) P( Bi ) P( A | Bi ) P( Bi )
P( Bi | A) = = k
P( A) å P( A | B j ) P( B j )
j =1

When P(A) is not directly known, but known conditionally,

we make use of this law.
26
 Bayes’ Rule for two events

P( A | B) P( B) P( A | B) P( B)
P( B | A) = =
P( A) P( A | B) P( B) + P( A | B) P( B)

• Now, solve Exercise 2.32, given P(B)

27
 Another example
• A novel disease diagnostic kit is 95% effective
in detecting a certain disease when it is
present. The test also has a 1% false positive
rate. If 0.5% of the population has the disease,
what is the probability a person with a positive
test result actually has the disease?

28
 Solution
• A = {a person’s test result is positive}
• B = {a person has the disease}
• P(B) = 0.005, P(A|B) = 0.95, P(A|Bc) = 0.01
P( A | B) P( B)
P( B | A) =
P( A | B) P( B) + P( A | B c ) P( B c )

0.95 ´ 0.005 475

= = » 0.323
0.95 ´ 0.005 + 0.01´ (1 - 0.005) 1470

29
 Random Variables
• A random variable (r.v.) associates a unique
numerical value with each outcome in the
sample space. It is a real-valued function from
a sample space Ω into real numbers.
• Similar to events it is denoted by an uppercase
letter (e.g., X or Y) and a particular value taken
by a r.v. is denoted by the corresponding
lowercase letter (e.g., x or y).

30
 Examples
• Toss three coins. X = number of heads
• Pick a student from the Computer Engineering
Department.
X = age of the student
• Observe lifetime of a light bulb
X = lifetime in minutes
• X may be discrete or continuous

31