DNSC 6221 JUD

Lecture Set 02
Bumsoo Kim
05/29/2012 The George Washington University School of Business Department of Decision Sciences

Introduction to Probability

Brief history of probability Probability interpretations Classical approach Relative frequency approach Subjective approach Definition of probability Laws of probability (basic principles) Conditional probability Statistical independence and mutually exclusive events Bayes Theorem Probability trees

Brief History of Probability

The use of probability as a concept to measure uncertainty dates back to hundreds of years ago. Probability has found applications in areas as diverse as medicine, gambling, weather forecasting and the law. It is generally believed that the mathematical theory of probability was started by the French mathematician Blaise Pascal (1623-1662) and Pierre Fermat (1601-1665) when they succeeded in deriving exact probabilities for certain gambling problems involving dice.

Probability Interpretations
1.

Frequency Approach: The probability of a particular outcome can be obtained if the process is repeated a large number of times under similar conditions. Ex: tossing a coin Classical Approach: The classical approach is a mathematical formula based approach in which the possible outcomes are enumerated and exact probabilities are calculated. Ex: rolling two dice

2.

Probability Interpretations

The classical approach would not work in processes where we cannot describe the possible outcomes and there is no way to determine the probabilities mathematically. Ex: Whether or not a project will be successful

3.

Subjective Approach: According to the subjective, or personal, interpretation of probability, the probability that a person assigns to a possible outcome of some process represents her own judgment of the likelihood that the outcome will be obtained (degree of belief). Ex: Expert opinions
5

Basic Principles
Note that the calculus of probability applies equally well no matter which interpretation one prefers.
Likelihood, chance Only consider random experiments Characteristics of random experiments Can specify all possible outcomes. Cannot predict a specific outcome with certainty.

Basic Principles
Ex: Todays closing price change of a security relative to yesterday.

Probability is the yardstick for measuring the uncertainty of these outcomes.

Venn Diagram of Outcomes

- Tool to represent probabilities - Used in set theory to show the mathematical or logical relationship between different sets

Outcomes of a roll of a die

Sample space: {1, 2, 3, 4, 5 ,6} Event A: outcome is even: {2, 4, 6} Event B: outcome is > 3: {4, 5, 6}

Basic Principles
Example:
An automobile dealer sells two brands of new cars. One, C, is primarily American in origin; the other, G, is primarily Japanese. The dealer performs repair work under warranty for both brands. Each warranty job is classified according to the primary problem to be fixed. If there is more than one problem in a given job, all problems are listed separately. Records for the past year indicate the following numbers of problems:
Engine Transmission Exhaust

Problem Area 211 115 326

Brand

C G Total

106 21 127

67 16 83

Fit/Finish

133 24 157

Other

24 6 30

Total

541 182 723

a.

What is the probability that a randomly chosen problem comes from brand C? P( randomly chosen problem comes from brand C) = .748
9

Basic Principles

Venn diagram

Mutually exclusive events: Only one of many events can occur at the same time

10

Basic Principles
Example:
Brand C G Total
Engine

106 21 127

Transmission Exhaust

Problem Area 211 115 326

67 16 83

Fit/Finish

133 24 157

Other

24 6 30

Total

541 182 723

b. Serious problems are those involving the engine or transmission. What is the probability that a randomly chosen problem is serious?
P (Serious problem) = P(E) + P(T) = 127/723 + 326/723 = 453/723 = .627

11

Addition Law of Probability

General Addition Law P(A or B) = P(A) + P(B) P(A and B)

A B

(A and B) has been counted twice Subtract once

Example: What is the probability that a randomly chosen problem comes from Brand C or is an engine problem?

C denotes "Brand C. P(C) = 541/723 E denotes Engine problem". P(E) =127/723 P(C or E) = P(C) + P(E) P(C and E) = 541/723 + 127/723 - 106/723 = 562/723 = .777

12

Complements Law

A
13

A` is also used to represent the complement of A.

Complements Law
Ex: Birthday Problem There are n people in a room. What is the probability that at least two of them have the same birthday? Assumptions: No birthdays fall on Feb 29; All 365 days are equally likely for each person; and Birthdays are independent A = {at least two people have same birthday} A` = {no two people have the same birthday} P (A` ) = (365)(364)(363)[365 (n-1)] / (365)n = .29 for n=30 so P(A)= 0.71 for n=50 P(A)=.97

14

Conditional vs Marginal Probability

Conditional Probability Conditional probability of event A occurring given that event B has occurred, denoted by P(A|B), is: P(A|B) = P(A and B) / P(B), provided P(B) > 0.

A and B
Marginal Probability: Probability of one variable taking a value irrespective of others, i.e; P(A), P(B)

15

Conditional vs Marginal Probability

Example:
Brand C G Total Engine 106 21 127 Problem Area Transmission Exhaust Fit/Finish 211 67 115 16 326 83 Other Total 133 24 24 6 157 30 541 182 723

a. For the auto dealers data, what is the probability that a randomly chosen problem is an engine problem, given that it comes from brand C?
P(Engine problem | Brand C) = P(E|C) = P(E and C)/P(C) = (106/723)/(541/723) =.196 where P(Brand C and Engine Problem) = P(C and E) = 106/723 and P(Brand C) = P (C) = 541/723

Note: P (E|C) = .196 while P(E) = .176 The occurrence of event C affects P(E)

16

Multiplication Rule

Multiplication Rule From the definition of conditional probability, P(A and B) = P(B) P(A|B) Obviously, P(A and B) = P(A) P(B|A)

Example: Randomly pick (without replacement) 2 cards from a standard deck. Find probability of 2 hearts. A = {1st card is a heart}, B = {2nd card is a heart} P (A and B) = P(A) P(B|A) = (13/52) (12/51)

The multiplication rule is useful in Probability Trees.

17

Statistical Independence

If occurrence of one event does not affect the other, they are independent events. Mathematically, A and B are independent events if and only if P(A|B) = P(A)

Example: Are events Brand C and Engine Problem independent? P(Engine Problem) = P(E) = 127/723 = .176 P(Engine Problem|Brand C) = P(E|C) = .196

Conclusion: The events Brand C and Engine Problem are not independent because P(E|C) P(E)
18

Statistical Independence

Multiplication Rule for Independent Events

If A and B are independent, then P(A and B) = P(A) P(B) Reason: From Multiplication Rule, P(A and B) = P(A|B) P(B) From independence, P(A|B) = P(A) P(A and B) = P(A) P(B)
19

Independent vs Mutually Exclusive Events

Coin flip example: One flip: Heads or Tails are mutually exclusive events Two flips: The outcome of each flip is independent If A and B are mutually exclusive events, then A and B can't both happen at the same time. Mutually exclusive events have no basic outcomes in common: P( A and B) = 0 If A and B are independent, then P(A and B) = P(A) P(B) Mutually exclusive events are always dependent.

20

Some Useful Definitions

Union of Events
The union of two events A and B is all basic outcomes that are part of either A or B or both: A B = A or B

Intersection of Events (Joint Probability)

The intersection of two events A and B is all basic outcomes that are part of both A and B: A B = A and B P( A and B) = P( A , B)

Formulas so far

Definition: 0 <= P (A) <= 1 Complements Law: P (A`)= 1- P (A) Addition Law: P (A or B) = P(A) + P(B) P(A,B) Multiplication Law: P (A and B) = P(B) P(A|B)

If A and B are independent: P (A and B) = P(A) P(B) Mutually exclusive events: P (A and B) = 0

22

Example
Example :

Suppose that A and B are mutually exclusive events. P(A) =0.4 and P(B) =0.3

Find P(A and B), P(A or B), P(not A and not B), P(A and not B), P(A|B), P(A|not B) and P(B|not A)

23

Probability Tables
- Tool to compute and represent probabilities - Ex: Bivariate Joint Probabilities
B1 A1
P(A1 B1)

B2
P(A1 B2)

... ...

Bk
P(A1 Bk)

A2
. Ah

P(A2 B1)

P(A2 B2)

...
. ...

P(A2 Bk)

.
P(Ah B1)

.
P(Ah B2)

.
P(Ah Bk)

Probability Trees

Many problems require the successive use of several of the basic principles to obtain a solution. These problems can be solved algebraically, but it is helpful to be familiar with some tools that can help you keep the logic straight. One of these tools is probability (or decision) trees. Probability trees use the multiplication law of probabilities.

In a probability tree, the probability for a specific path is found by using the multiplication rule.
25

Probability Trees
Exercise: A purchasing dept. finds that 75% of its special orders are received on time. Of those orders that are on time, 80% meet specifications completely; of those orders that are late, 60% meet them. T = {Order is on time} P(T) = .75 P(M|T) = .80 M = {Meets specifications} P (M| not T ) = .60

26

Probability Trees
P(M|T) = P(T) = .75 .80 P(T and M) = .60 P(T and M ) = .15 P(T and M) = .15 P(T and M ) = .10

P(M |T) =
P(M|T ) = P(T ) = .25 P(M | T ) =

.20
.60 .40

27

Probability Trees

Find the probability that an order is on time and meets specifications. P(T and M) = .60

Find the probability that an order meets specifications. P(M) = P(M and T) + P(M and not T) = .60 + .15 = .75

28

Probability Trees
Exercise:

A manufacturer of snack crackers introduces several new products each year. About 60% of the introductions are failures, 30% are moderate successes, and 10% are major successes. To try to improve the odds, the manufacturer tests new products in a customer tasting panel. Of the failures, 50% receive a poor rating in the panel, 30% a fair rating and 20% receive a good rating, For the moderate successes, 20% receive a poor rating, 40% a fair rating, and 40% a good rating. For major successes, the percentages are 10% poor, 30% fair and 60% good.

29

Probability Trees

Find the joint probability of a new product being a failure and receiving a poor rating.
Construct a probability table of all possible joint probabilities of new product results and panel rating. If a new product receives a good rating, what is the probability that the product will be a failure? Create a probability tree using the given information. Use the tree to find the probability that a new product will be a major success, given that it gets a poor rating.

30

Bayes Theorem

Extension of Conditional Probability Result generalizes to M events, A1, A2, ..., AM

P A1 | B

P A1 P B

B P B | A1 P A1 P B | A2 P A2
31

P B | A1

P A1

Bayes Theorem

Importance: Update of prior beliefs

P (A1) prior belief about probability of A1 P (B | A1) probability relating information / signal to outcome

(based on information about past relationship between signal and outcome) Bayes Theorem provides means to calculate posterior probability P (A1 | B)

32

Example: Market Research at Arrow Cosmetics

Arrow Cosmetics is considering the introduction of a new location Possible outcomes: Units sold = {600K, 300K, 200K} = {High, Medium, Low} Subjective Probabilities:

High Medium Low

0.3 0.4 0.3

33

Additional Information

Arrow can hire Marble Research to perform market study test for new formula Possible evaluations: {Hit, Flop} Past experience with Marbles research: P (Hit | High) = 1.0 P (Hit | Medium) = 0.375 P (Hit | Low) = 0.166

34

Revising Probabilities

Updating the a priori/marginal probability with additional information (signal,test,symptom) to come up with revised/posterior/conditional probability Suppose that Marble declares that the new product will be a hit Should Arrow adjust its belief about the probability that sales will be high? If so, how? P(High) Test P (High | Hit) P(Medium) Test P (Medium | Hit) P(Low) Test P (Low | Hit)

35

Lets Make a Deal!

Suppose you are on a game show, and you are given the choice of three doors. Behind one door is a car, behind the others, goats. You pick a door, say number 1, and the host, who knows what's behind the doors, opens a door which has a goat. Say this is door 3. The host asks you: "Do you want to pick door number 2?" Is it to your advantage to switch your choice of doors? http://www.youtube.com/watch?v=mhlc7peGlGg http://www.youtube.com/watch?v=P9WFKmLK0dc&feature=fvw

36

Summary

Interpretations of probability (classical, frequency and subjective approaches)

Rules and definitions of probability; The addition law for mutually exclusive events and non mutually exclusive events, the complements law of probability. Marginal, joint and conditional probabilities

Statistical Independence
Probability Trees
37

Lecture 3

Topics :

- Random variables - Probability distributions, joint and marginal distributions - Expected value, standard deviation (or variance) of a random variable - Covariance & Correlation

Things to do :
Go over these notes & Check Ch. 4.1, 4.2 for clarifications Read Chapter 4.3-4.9 in the textbook Attempt Questions 3-4 in Assignment 1

38