Lecture 8 Tutorial Questions

Exercise 1:

A simulation model of a job shop was developed to investigate different scheduling rules.
To validate the model, the scheduling rule currently used was incorporated into the model
and the resulting output was compared against observed system behavior. By searching
the previous year’s database records, it was estimated that the average number of jobs in
the shop was 22.5 on a given day. Seven independent replications of the model were run,
each of 30 days’ duration, with the following results for average number of jobs in the shop:

18.9; 22.0; 19.4; 22.1; 19.8; 21.9; 20.2

a) Develop and conduct a statistical test to evaluate whether model output is

consistent with system behavior. Use level of significance 𝛼 = 0.05.
b) What is the power of this test if a difference of two jobs is viewed as critical? What
sample size is needed to guarantee a power of 0.8 or higher? (Use 𝛼 = 0.05)

ANSWER

a) Population mean = 𝜇0 = 22.5 days; n = 7; use t-distribution with n-1 degrees of

freedom

1 18.9+22.0+19.4+22.1+19.8+21.9+20.2
Sample mean = 𝑌̅ = 𝑛 ∑𝑛𝑖=1 𝑌 = = 20.614
7

∑𝑛 ̅ 2
𝑖=1(𝑌− 𝑌 )
Sample Standard Deviation = 𝑆𝑌 = = 1.36
𝑛−1

Test of significance at 𝛼 = 0.05

𝐻0 : E(Y) = 22.5
𝐻1 : E(Y) ≠ 22.5

𝑌− 𝑢0 20.614− 22.5
|𝑡𝛼,𝑛−1 |= 𝑆/ = |𝑡0.025,6 |= = -3.67
2 √𝑛 1.36/√6

Since |𝑡0.025,6 | > 2.45 we reject the null hypothesis and conclude that the model is
not okay or inadequate.
 b) |𝐸(𝑌) − 𝜇0 | = Critical Difference Allowed = 2 jobs

|𝐸(𝑌)− 𝜇0 | |𝐸(𝑌)− 𝜇0 | 2
𝛿= =𝛿= = 1.36 = 1.47
𝜎 𝑆
𝛽(𝛿̂ ) = 𝛽(1.47) = 0.1 for n = 7
Power = 1 – 𝛽 = 1 – 0.10 = 0.90

Sample size needed for the required Power = 0.80 or 𝛽 ≤ 0.20

Assuming our S = 1.36
Then for 𝛼 = 0.05 and 𝛿 1.47, n = 6

 If you calculate or approximate a sample size and you feel is too high for evaluating
the hypothesis of the difference between the model statistics and the population
parameter, you either need a higher 𝛽 risk (lower power) or a larger difference
(critical difference) 𝛿 must be considered.

Exercise 2:

System data for the job shop of Exercise 1 revealed that the average time spent by a job in
the shop was approximately 4 working days. The model made the following predictions, on
seven independent replications, for average time spent in the shop:

3.70; 4.21; 4.35; 4.13; 3.83; 4.32; 4.05

a) Is model output consistent with system behavior? Conduct a statistical test, using
level of significance 𝛼 = 0.01.
b) If it is important to detect a difference of 0.5 day, what sample size is needed to have
a power of 0.90? Interpret your results in terms of model validity or invalidity. (Use
𝛼 = 0.01)

ANSWER

a) Population mean = 𝜇0 = 4 days; n = 7; use t-distribution with n-1 degrees of

freedom; Test of significance at α = 0.01

1 3.70+4.21+4.35+4.13+3.83+4.32+4.05
Sample mean = 𝑌̅ = 𝑛 ∑𝑛𝑖=1 𝑌 = = 4.084
7

∑𝑛 ̅ 2
𝑖=1(𝑌− 𝑌 )
Sample Standard Deviation = 𝑆𝑌 = = 0.2441
𝑛−1

𝐻0 : E(Y) = 4
𝐻1 : E(Y) ≠ 22.5
 𝑌− 𝑢0 20.614− 22.5
|𝑡𝛼,𝑛−1 |= 𝑆/ = |𝑡0.005,6 |= = 0.91
2 √𝑛 1.36/√6

Since |𝑡0.005,6 | < 3.71 we do not reject the null hypothesis and conclude that the
model is okay or is adequate.

b) |𝐸(𝑌) − 𝜇0 | = Critical Difference Allowed = 0.5 days

|𝐸(𝑌)− 𝜇0 | |𝐸(𝑌)− 𝜇0 | 0.5

𝛿= =𝛿= = 0.2441 = 2.05
𝜎 𝑆
𝛽(𝛿̂ ) = 𝛽(2.05) = 0.1 for n = 7
Power = 1 – 𝛽 = 1 – 0.10 = 0.90

 Then, assuming that the population standard deviation is 0.2441, the current power
of the test is 0.90.

Exercise 3:

For the job shop of Exercise 1, four sets of input data were collected over four different 10-
day periods, together with the average of jobs in the shop 𝑍𝑖 for each period. The input data
were used to drive the simulation model for four runs of 10 days each, and model
predictions of average number of jobs in the shop 𝑌𝑖 were collected, with these results:

i 1 2 3 4
𝑍𝑖 21.7 19.2 22.8 19.4
𝑌𝑖 24.6 21.1 19.7 24.9

a) Conduct a statistical test to check the consistency of system output and model
output. Use the level of significance 𝛼 = 0.05.
b) If a difference of two jobs is viewed as important to detect, what sample size is
required to guarantee a probability of at least 0.80 of detecting this difference, if it
indeed exists? (Use 𝛼 = 0.01)

ANSWER
1
a) Average observed difference = 𝑑̅ = 𝐾 = ∑𝐾
𝑗=1 𝑍𝑖𝑗 − 𝑌𝑖𝑗 = 1.80
1
Squared Deviation from Mean = 𝑆𝑑2 = 𝐾−1 ∑𝐾 ̅ 2
𝑗=1(𝑑𝑗 − 𝑑 ) = 12.96
Standard Deviation = 𝑆𝑑 = 3.60

Testing for significance at 𝛼 = 0.05

 𝐻0 : 𝜇𝑑 = 0
𝐻1 : 𝜇𝑑 ≠ 0

𝑑̅− 𝑢𝑑 1.80− 0
|𝑡𝛼,𝑛−1 |= = |𝑡0.025,3 |=3.60/√4 = 1.0
2 𝑆/√𝑛

Since |𝑡0.025,3 | < 3.18 we do not reject the null hypothesis and conclude that the
model is okay or is adequate.

b) |𝐸(𝑌) − 𝜇0 | = Critical Difference Allowed = 2 jobs

|𝐸(𝑌)− 𝜇0 | |𝐸(𝑌)− 𝜇0 | 2
𝛿= =𝛿= = 3.60 = 0.556
𝜎 𝑆

Sample size needed for the required Power = 0.80 or 𝛽 ≤ 0.20

Assuming our S = 3.60
Then for 𝛼 = 0.05 and 𝛿 = 0.556, n = 30