Regression Analysis: Answers To Problems and Cases 1. 2
Regression Analysis: Answers To Problems and Cases 1. 2
Regression Analysis: Answers To Problems and Cases 1. 2
REGRESSION ANALYSIS
1. Option b is inconsistent because the regression coefficient and the correlation coefficient
must have the same sign.
Analysis of Variance
Source DF SS MS F P
Regression 1 92432 92432 20.47 0.002
Residual Error 8 36121 4515
Total 9 128552
a. Yes, the regression is significant. Reject using either the t value 2.384
and it’s p value .002, or the F ratio 20.47 and it’s p value .002.
b. Y = 828 + 10.8X
94
f. Total variation (SST) is 128,552
Analysis of Variance
Source DF SS MS F P
Regression 1 25.622 25.622 115.52 0.000
Residual Error 8 1.774 0.222
Total 9 27.396
a. Yes, the regression is significant. Reject using either the t value 10.75
and it’s p value .000, or the F ratio 115.52 and it’s p value .000.
b. Y = .620 + .109X
2
1 (X X )
1
n ( X X )2
sf = sy.x
sf = .471(1.110) = .523
95
small sample size and large confidence coefficient. Not useful.
5. a, b and d.
Analysis of Variance
Source DF SS MS F P
Regression 1 634820 634820 50.96 0.000
Error 7 87197 12457
Total 8 722017
e. Reject at the 5% level since F = 50.96 and it’s p value = .000 < .05.
Could also use t = 7.14, the t value associated with the slope coefficient, and it’s
p value = .000. The correlation coefficient is significantly different from 0 since
the slope coefficient is significantly different from 0.
96
6. a, b and d.
Analysis of Variance
Source DF SS MS F P
Regression 1 27032.3 27032.3 83.74 0.000
Error 9 2905.4 322.8
Total 10 29937.6
e. Reject at the 10% level since F = 83.74 and it’s p value = .000 < .10.
Could also use t = 9.15, the t value associated with the slope coefficient, and it’s
p value = .000. The correlation coefficient is significantly different from 0 since
the slope coefficient is significantly different from 0.
97
f. Based on the residuals versus the fitted values plot, there is no reason to
doubt the adequacy of the simple linear regression model.
7. a, b, c & d.
The regression equation is
Orders = 15.8 + 1.11 Catalogs (Fitted regression line)
Source DF SS MS F P
Regression 1 317.53 317.53 9.58 0.011
Residual Error 10 331.38 33.14
Total 11 648.92
New
Obs Fit SE Fit 90% CI 90% PI
1 26.98 1.93 (23.47, 30.48) (15.97, 37.98)
98
e. Do not reject at the 1% level since t = 3.10 and it’s p value = .011 > .01.
However, would reject at the, say, 5% level.
f. Do not reject at the 1% level since F = 9.58 and it’s p value = .011 > .01.
Result is consistent with the result in e as it should be.
g. See Fit and 90% PI at end of computer printout above. A 90% prediction interval
for mail orders when 10(000) catalogs are distributed is (16, 38)---16,000 to 38,000.
Analysis of Variance
Source DF SS MS F P
Regression 1 978986 978986 7.69 0.024
Residual Error 8 1017824 127228
Total 9 1996810
a. There is a significant (at the 5% level) negative relationship between these variables.
Reject at the 5% level since t = -2.77 and it’s p value = .024 < .05.
b. The data set is small. Moreover, r2 = .49 so only 49% of the variation in investment
dollars is explained by interest rate. Finally, the last observation (6.2, 1420) has a
large influence on the location of the fitted straight line. If this observation is deleted,
there is a considerable change in the slope (and intercept) of the fitted line. Using the
original straight line equation for prediction is suspect.
c. A forecast can be calculated. It is 1865. However, the 95% prediction interval is wide.
Forecast unlikely to be useful without additional information. See comments in b.
d. See answer to b.
e. It seems reasonable to say movements in interest rate cause changes in the level
of investment.
9. a. The firms seem to be using very similar rationale since r = .959. Also, from the fitted
99
line plot below, notice the fitted line is not far from the 45 o line through the origin (with
intercept 0 and slope 1).
b. If ABC bids 1.01, the predicted competitor’s bid is 101.212. A 95% prediction
interval (PI) is given below.
New
Obs Fit SE Fit 95% CI 95% PI
101 101.212 0.164 (100.872, 101.552) (99.637, 102.786)
c. Assume normality distributed errors about the population regression line and
treat the least square line as if it were the population regression line (n is reasonably
large in this case). Then at ABC bid 101, possible competitor bids are normally
distributed about the fitted value 101.212 with a standard deviation estimated by
sy.x = .743. Consequently, the probability that ABC will have the bid is
P(Z ≥ (101-101.212)/ .743) = P(Z ≥ -.285) = .51.
10. a. Only if the sample size is large enough. The t statistic associated with the
slope coefficient or the F ratio should be consulted to determine if the population
regression line slope is significantly different from a horizontal line with zero
slope.
100
b. The regression equation is
Permits = 2217 - 145 Rate
Analysis of Variance
Source DF SS MS F P
Regression 1 559607 559607 26.88 0.001
Residual Error 7 145753 20822
Total 8 705360
c. Reject at the 5% level since t = -5.18 and it’s p value = .001 < .05.
d. If interest rate increases by 1%, on average the number of building permits will
decrease by 145.
f. Interest rate explains about 79% of the variation in number of building permits issued.
Analysis of Variance
Source DF SS MS F P
Regression 1 14331 14331 231.77 0.000
Residual Error 11 680 62
Total 12 15011
c. Reject at the 5% level since t = 15.22 and it’s p value = .000 < .05
d.
102
Residual Versus Fits plot shows curvature in scatter not captured by straight line fit.
e. Model with quadratic term in Batch Size fits well. Results with Size**2 as
predictor variable follow.
Analysis of Variance
Source DF SS MS F P
Regression 1 14951 14951 2727.00 0.000
Residual Error 11 60 5
Total 12 15011
f. Reject at the 5% level since t = 52.22 and it’s p value = .000 < .05
103
h. Predicted Values for New Observations
New
Obs Fit SE Fit 95% CI 95% PI
1 95.411 1.173 (92.829, 97.993) (89.647, 101.175)
14. a.
104
c. . About 38% of the variation in market prices is explained by
assessed values (as predictor variable). There is a considerable amount of
unexplained variation.
Unusual Observations
f. Unusual Observations
Obs PlayCosts OpExpens Fit SE Fit Residual St Resid
7 18.0 60.00 42.31 1.64 17.69 3.45R
Analysis of Variance
106
Source DF SS MS F P
Regression 1 10855642 10855642 16.15 0.001
Residual Error 21 14113925 672092
Total 22 24969567
Although the regression is significant, the residual versus fit plot indicates the
magnitudes of the residuals increase with the level. This behavior and the
scatter diagram in a suggest that consumption is not evenly distributed about
the regression line. That is, the data have a megaphone-like appearance. A
straight line regression model for these data is not adequate.
c & d. The response variable is converted to the natural log of newsprint consumption
(LnConsum).
Analysis of Variance
Source DF SS MS F P
Regression 1 3.8252 3.8252 16.00 0.001
Residual Error 21 5.0209 0.2391
107
Total 22 8.8461
The regression is significant (F = 16, p value = .001) although only 43% of the
variation in ln(consumption) is explained by families. The residual plots
above suggest the straight line regression of ln(consumption) on families is
adequate. This simple linear regression model with ln(consumption) is better
than the same model with consumption as the response.
17. a. Can see from fitted line plot below that growth in number of steakhouses is
exponential, not linear.
108
b. The slope of a regression of ln(location) versus year is related to the annual
growth rate.
Analysis of Variance
Source DF SS MS F P
Regression 1 11.764 11.764 82.91 0.001
Residual Error 4 0.568 0.142
Total 5 12.332
18. a, Can see from fitted line plot below that growth in number of copy centers is
exponential, not linear.
109
b. The slope of a regression of ln(centers) on time (year) is related to the annual
growth rate.
Analysis of Variance
Source DF SS MS F P
Regression 1 53.078 53.078 1476.38 0.000
Residual Error 12 0.431 0.036
Total 13 53.509
b. Cannot reject H0 at the 10% level since the t value associated with the slope
coefficient, –1.57, has a p value of .138 > .10. The regression is not significant.
110
There does not appear to be a relationship between profits per employee and
number of employees.
c. r2 = .15. Only 15% of the variation in profits per employee is explained by the
number of employees.
d. The regression is not significant. There is no point in using the fitted function to
generate forecasts for profits per employee for a given number of employees.
Analysis of Variance
Source DF SS MS F P
Regression 1 579.40 579.40 5.99 0.029
Residual Error 13 1258.40 96.80
Total 14 1837.80
The regression is now significant at the 5% level (t value = -2.45, p value = .029 < .05).
r2 has increased from 15% to 31.5%. These results suggest there is a linear
relationship between profits per employee and number of employees. A single
observation can have a large influence on the regression analysis, particularly when
the number of observations is relatively small. However, the relatively small r2 of 31.5%
indicates there will be a fair amount of uncertainly associated with any forecast of
profits per employee. Dun and Bradstreet should not be thrown out unless there is some
good (non-numerical) reason not to include this firm with the others.
111
Constant 0.683 1.691 0.40 0.690
Estimate 0.92230 0.08487 10.87 0.000
Analysis of Variance
Source DF SS MS F P
Regression 1 3833.4 3833.4 118.09 0.000
Residual Error 24 779.1 32.5
Total 25 4612.5
e. The plot of the residuals versus the fitted values has a megaphone-like appearance.
The residuals are numerically smaller for smaller projects than for larger projects.
Estimated costs are more accurate predictors of actual costs for inexpensive (smaller)
projects than for expensive (larger) projects.
22. a. The regression is significant (t value = 14.71, p value = .000).
112
c. If ln(estimated costs) are perfect predictor of ln(actual costs), then .
The estimated intercept coefficient, .003, is consistent with . With the
t value = .02 and its p value = .987, cannot reject the null hypothesis .
To check the hypothesis compute t =(.968-1)/.0658 = –.49, which is not
in the rejection region for a two-sided test at any reasonable significance level.
The estimated slope coefficient, .968, is consistent with .
d. ln(24) = 3.178, so forecast of ln(actual cost) = .0026 + .968(3.178) = 3.079. Forecast
of actual cost is e3.079 = 21.737.
This case asks students to summarize the analysis in a report to management. We find this a
useful exercise since it requires students to put the application and results of a statistical procedure into
their own words. If they are able to do this, they understand the technique.
This case illustrates the use of regression analysis in a situation where determining a good
regression equation is only the first step. The results must then be priced out in order to
arrive at a rational decision regarding a pricing policy. This situation can generate a discussion
regarding the general nature of quantitative techniques: they aid in the decision-making
process rather than replace it. Possible policies regarding the small-load charge can be
discussed after the cost of such loads is determined. One approach would be to take small loads
at company cost, which is low. The resultant goodwill might pay off in increased regular
business. Another would be to charge a low cost for small loads but only if the customer agrees to
book a certain number of large loads.
The low out-of-pocket cost involved in adding small loads can focus management attention
in other directions. Since no significant costs need to be recovered by the small load charge,
a policy based on other considerations is appropriate.
1. The 89 degree temperature is 24 degrees off ideal (89 - 65 = 24). This value is placed into
the regression equation yielding a forecast number of units per day of 338.
2. Once again, the temperature is 24 degrees from ideal (65 - 41 = 24). For X = 24, a forecast
of 338 units is calculated from the regression equation.
3. Since there is a fairly strong relationship between output and deviation from ideal
temperature (r = -.80), higher output may well result from efforts to control the
temperature in the work area so that it is close to 65 degrees. Gene should consider ways
to do this.
4. Gene has made a decent start towards finding an effective forecasting tool. However,
since about 36% of the variation in output is unexplained, he should look for additional
important predictor variables.
2. Y = –4.28 + .254X
3. r2 = .859. About 86% of Y's (absent days) variability can be explained through
knowledge of X (employee age).
4. The null hypothesis is rejected using either t = 8.9, p value = .000 or the
F = 79.3 with p value = .000. There is a significant relation between absent days and
employee age.
5. Placing X = 24 into the prediction equation yields a Y forecast of 1.8 absent days per year.
6. If time and cost are not factors, it might be helpful to take a larger sample to see if these
small sample results hold. If results hold, a larger sample will very likely produce
more precise interval forecasts.
7. The fitted function is likely to produce useful forecasts, although 95% prediction
intervals can be fairly wide because of the small sample size.
1. After John uses simple regression analysis to forecast his monthly sales volume, he is
not satisfied with the results. The low r-squared value (56.3%) disappoints him.
The high seasonal variation should be discussed as a cause of his poor fit
when using only the month number to forecast sales. The possibility of using
dummy variables to account for the monthly effect is a possibility. After this topic
is covered in Chapter 7, you can have the students return to this case.
2. Not adequate.
3. The idea of serial correlation can be mentioned at this point. The possibility of
autocorrelated residuals can be introduced based on John's Durbin-Watson statistic.
In fact, the DW is low, indicating definite autocorrelation. A class discussion about
this problem and what might be done about it is useful. After this topic is covered
in Chapter 8, you can have the students return to this case. We hope that by this
time students appreciate the difficulties involved in real-life forecasting. Forecasting
Compromises and multiple attempts are the norm, not exceptions.
114
CASE 6-5: CONSUMER CREDIT COUNSELING
Analysis of Variance
Source DF SS MS F P
Regression 1 5891.9 5891.9 10.51 0.002
Residual Error 46 25791.4 560.7
Total 47 31683.2
The correlation of Clients and Index = 0.752. The relation is significant (see below).
Analysis of Variance
Source DF SS MS F P
Regression 1 49993 49993 126.04 0.000
Residual Error 97 38475 397
Total 98 88468
2. The regression equation is Clients = - 199 + 2.94 BI
115
Note: Students might develop a new equation that leaves out the first three months of
data for 1993. This is a better way to determine whether the model works and the
results are:
Analysis of Variance
Source DF SS MS F P
Regression 1 43028 43028 107.52 0.000
Residual Error 94 37617 400
Total 95 80645
Regressing Clients on the reciprocal of Index produces a little better straight line fit.
The results for this transformed predictor variable follow.
Analysis of Variance
Source DF SS MS F P
Regression 1 45015 45015 118.76 0.000
Residual Error 94 35630 379
Total 95 80645
3. Actual Forecast Forecast Forecast(RecipIndex predictor)
6. If a good regression equation can be developed in which the changes in the predictor
variable lead the response, it might be possible to accurately forecast the rest of 1993.
However, if the regression equation is based on coincident changes in the predictor
variable and response, forecasts for the rest of 1993 could not be developed since values
for the predictor variable are not known in advance.
1. The four linear regression models are shown below. Both temperature and rainfall are
potential predictor variables.
2. & 3. Sixty-five degrees was subtracted from the temperature variable. The variable used
was the absolute value of the temperature with relative zero at 65 degrees Fahrenheit
labeled NewTemp.
The correlation coefficient between Calls and NewTemp is .724, indicating a fairly
strong positive linear relationship. However, examination of the fitted line plot below
suggests there is a curvilinear relation between Calls and NewTemp
118
Predictor Coef SE Coef T P
Constant 20044.4 203.1 98.68 0.000
NewTemp**2 5.3817 0.5462 9.85 0.000
Analysis of Variance
Source DF SS MS F P
Regression 1 119870408 119870408 97.08 0.000
Residual Error 55 67910916 1234744
Total 56 187781324
CHAPTER 7
MULTIPLE REGRESSION
119
1. A good predictor variable is highly related to the dependent variable but not too
highly related to other predictor variables.
2. The population of Y values is normally distributed about E(Y), the plane formed by the
regression equation. The variance of the Y values around the regression plane is
constant. The residuals are independent of each other, implying a random sample. A
linear relationship exists between Y and each predictor variable.
3. The net regression coefficient measures the average change in the dependent variable per
unit change in the relevant independent variable, holding the other independent variables
constant.
b. The entries in a correlation matrix reflected about the main diagonal are the
same. For example, r32 = r23.
120
e. Yes. Variables 5 and 6 are to some extent collinear, r56 = .69.
f. Models that include variables 4 and 6 or variables 2 and 5 are possibilities. The
predictor variables in these models are related to the dependent variable and not
too highly related to each other.
g. Variable 5.
8. a. Correlations:
Time Amount
Amount 0.959
Items 0.876 0.923
Analysis of Variance
Source DF SS MS F P
Regression 2 128.988 64.494 87.71 0.000
Residual Error 15 11.030 0.735
Total 17 140.018
Amount and Time are highly collinear (correlation = .923, VIF = 6.756). Both
variables are not needed in the regression function. Deleting Items with the
non-significant t value gives the best regression below.
121
Analysis of Variance
Source DF SS MS F P
Regression 1 128.90 128.90 185.54 0.000
Residual Error 16 11.12 0.69
Total 17 140.02
b. From the Full Model, checkout time decreases by .039 which does not
make sense.
g. Using the best model, the 95% prediction interval (interval forecast) for
Amount = $70 is given below.
New
Obs Fit SE Fit 95% CI 95% PI
1 6.008 0.238 (5.504, 6.512) (4.171, 7.845)
Food Income
Income 0.884
Size 0.737 0.867
When income is increased by one thousand dollars holding family size constant, the
average increase in annual food expenditures is 228 dollars. When family size is
increased by one person holding income constant, the average decrease in annual
food expenditures is 41 dollars. Since family size is positively related to food
expenditures, r = .737, it doesn’t make sense that a decrease in expenditures
would occur.
10. a. Both high temperature and traffic count are positively related to number of six-
packs sold and have potential as good predictor variables. There is some collinearity
(r = .68) between the predictor variables but perhaps not enough to limit their
value.
t= = = 3.45
Reject H0 because 3.45 > 2.898 and conclude that the regression coefficient for
the high temp-variable is unequal to zero in the population.
.06795
t= = = 3.35
.02026
Reject H0 because 3.35 > 2.898 and conclude that the regression coefficient for
the traffic count variable is unequal to zero in the population.
2727.9
e. sy.x’s = = = 160.46 = 12.67
( 20 3)
f. If there is an increase of one degree in high temperature while the traffic count
is held constant, beer sales increase on an average of .78 six-packs.
g. The predictor variables explain 81% of the variation in six-packs sold. Both
predictor variables are significant. It would be prudent to examine the residuals (not
available in the problem) before deciding to use the fitted regression function for
forecasting however.
11. a. Scatter diagram follows. Female drivers indicated by solid circles, male divers by
diamonds.
124
b. The regression equation is: = 25.5 - 1.04 X1 + 1.21 X2
For a given age of car, female drivers expect to get about 1.2 more miles
per gallon than male drivers.
d.
Sales Outlets
125
Outlets 0.739
Auto 0.548 0.670
Number of retail outlets is positively related to annual sales, r12 = .74, and is
potentially a good predictor variable. Number of automobiles registered is
moderately related to annual sales, r13 = .55, and is positively correlated with
number of retail outlets, r23 = .67. Given number of retail outlets in the
regression function, number of automobiles registered may not be required.
Analysis of Variance
Source DF SS MS F P
Regression 2 1043.7 521.8 4.91 0.041
Residual Error 8 849.6 106.2
Total 10 1893.2
New
Obs Fit SE Fit 95% CI 95% PI
1 37.00 7.15 (20.50, 53.49) (8.07, 65.93)
As can be seen from the regression output, it appears as if each predictor variable is
not significant (at the 5% level), however the regression is significant at the 5%
level. This is one of things that can happen when the predictor variables are collinear.
The forecast for region 1 is 37 with a prediction error of 52.3 – 37 = 15.3. However,
it is not a good idea to use this fitted function for forecasting. If the regression is rerun
after deleting Auto, Outlets (and the regression) is significant at the 1% level and
126
c. Y = 10.11 + .011(2500) + .195(20.2) = 41.549 (million)
d. The standard error of estimate is 10.3 which is quite large. As explained in part b,
the fitted function with both predictor variables should not be used to forecast.
Even if the regression is rerun with the single predictor Outlets, R2 =55% and
the relatively large standard error of the estimate suggest there will be a lot of
uncertainly associated with any forecast.
e. sy.x’s = = = = 10.3
f. If one retail outlet is added while the number of automobiles registered remains
constant, sales will increase by an average of .011 million or $11,000 dollars. If
one million more automobiles are registered while the number of retail outlets
remains constant, sales will increase by an average of .195 million or $195,000
dollars. However, these regression coefficients are suspect due to collinearity
between the predictor variables.
g. New predictor variables should be tried.
Analysis of Variance
Source DF SS MS F P
Regression 3 1843.40 614.47 86.32 0.000
Residual Error 7 49.83 7.12
Total 10 1893.23
127
Personal income by region makes a significant contribution to sales. Adding Income
to the regression function results in an increase in R2 from 55% to 97%. In addition,
the t value and corresponding p value for Income indicates the coefficient of this
variable in the population is different from 0 given predictor variables Outlets and
Sales. Notice however, the regression should be rerun after deleting the insignificant
predictor variable Outlets. The correlation matrix and the VIF numbers suggest
Outlets is multicollinear with Auto and Income.
New
Obs Fit SE Fit 95% CI 95% PI
1 27.306 1.878 (22.865, 31.746) (19.591, 35.020)
New
Obs Outlets Auto Income
1 2500 20.2 40.0
c. The standard error of estimate has been reduced to 2.67 from 10.3 and R2 has increased
to 97%. The 95% PI in part b is fairly narrow. The forecast for region 12 sales in
part be should be accurate.
d. The best choice is to drop Outlets from the regression function. If this is done,
the regression equation is
Sales = - 4.03 + 0.621 Auto + 0.430 Income
Measures of fit are nearly the same as those for the full model and there is no longer
a multicollinearity problem.
14. a. Reject H0 : 1 = 0 if |t |> 3.1.
.65
t= = 13
.05
Reject H0 and conclude that the regression coefficient for the aptitude test variable
is significantly different from zero in the population.
Similarly, Reject H0 : 2 = 0 if |t |> 3.1.
128
20.6
t= = 12.2
1.69
Reject H0 and conclude that the regression coefficient for the effort index variable
is significantly different from zero in the population.
b. If the effort index increases one point while aptitude test score remains constant,
sales performance increases by an average of $20.600.
c. Y = 16.57 + .65(75) + 20.6(.5) = 75.62
(Y Y ) 2 139.4
f. R = 1 -
2
2 = 1 - = 1 - .039 = .961
(Y Y ) 3569.3
g.
15. a. Scatter plot for cash purchases versus number of items (rectangles) and credit card
purchases versus number of items (solid circles) follows.
129
b. Minitab regression output:
Notice that for a given number of items, sales from cash purchases are estimated to
be about $18.60 less than gross sales from credit card purchases.
c. The regression in part b is significant. The number of items sold and whether
the purchases were cash or credit card explains approximately 83% of the
variation in gross sales. The predictor variable Items is clearly significant. The
coefficient of the dummy variable X2 is significantly different from 0 at the
130
10% level but not at the 5% level. From the residual plots below we see that
there are a few large residuals (see, in particular, cash sales for day 25 and credit
card sales for day 1); but overall, plots do not indicate any serious departures
from the usual regression assumptions.
f. Fitted function in part b is effectively two parallel straight lines given by the
equations:
Cash purchases: Y = 13.61 + 5.99Items – 18.6(1) = -4.98 + 5.99Items
Credit card purchases: Y = 13.61 + 5.99Items
b. The stepwise results are the same for an alpha to enter = alpha to remove = .05 or
.15 (the Minitab default) or F to remove = F to enter =4.
Step 1 2
Constant 20.40 71.23
ERA -18.0
T-Value -9.52
P-Value 0.000
S 7.72 3.55
R-Sq 39.28 87.72
The fitted function from the stepwise program is:
17. a. View will enter the stepwise regression function first since it has the largest
correlation with Price. After that the order of entry is difficult to determine from
132
the correlation matrix alone. Several of the predictor variable pairs are fairly highly
correlated so multicollinearity could be a problem. For example, once View is in the
model, Elevation may not enter (be significant). Slope and Area are correlated so
it may be only one of these predictors is required.
Analysis of Variance
Source DF SS MS F P
Regression 3 3071.1 1023.7 5.29 0.010
Residual Error 16 3096.7 193.5
Total 19 6167.8
Unusual Observations
New
Obs Fit SE Fit 95% CI 95% PI
1 80.88 3.36 (73.77, 88.00) (50.55, 111.22)
F = 5.29 with a p value = .010, so the regression is significant at the 1% level.
The predicted final exam score for within term exam scores of 86 and 77 and a
GPA of 3.4 is
133
The variance inflation factors (VIF’s) are all small (near 1); however, the t ratios and
corresponding p values suggest that each of the predictor variables could be dropped
from the regression equation. Since the F ratio was significant, we conclude that
multicollinearity is a problem.
d. Mean leverage = (3+1)/20= .20. None of the observations are high leverage points.
e. From the regression output above, observation 20 has a large standardized residual.
The fitted model over-predicts the response (final exam score) for this student.
19. Stepwise regression results, with significance level .05 to enter and leave the
regression function, follow.
Step 1
Constant -26.24
X3 31.4
T-Value 3.30
P-Value 0.004
S 14.6
R-Sq 37.71
R-Sq(adj) 34.25
The “best” regression model relates final exam score to the single predictor
variable grade point average.
Predictor
Variables
X1 .295
X2 .301
X3 .377
X1, X2 .404
X1, X3 .452
X2, X3 .460
X1, X2, X3 .498
The criterion would suggest using all three predictor variables. However, the
results in problem 7.18 suggest there is a multicollinearity problem with three
predictors. The best two independent variable model uses predictors X2 and X3.
When this model is fit, X2 is not required. We end up with a model involving the
single predictor X3, the model selected by the stepwise procedure.
134
20. Best three predictor variable model selected by stepwise regression follows.
Unusual Observations
Obs Educate LnComp Fit SE Fit Residual St Resid
31 2.00 6.5338 5.9055 0.4386 0.6283 2.73RX
33 0.00 6.3969 7.0645 0.2624 -0.6676 -1.59 X
All in all, this k = 3 predictor model appears to be better than the k = 2 predictor
model of Example 7.12.
135
a. & b. The regression equation is
Analysis of Variance
Source DF SS MS F P
Regression 2 51130 25565 165.95 0.000
Residual Error 7 1078 154
Total 9 52208
The coefficient of Accounts changes from the quadratic model to the straight
line model because, not surprisingly, Accounts and Accounts**2 are highly
collinear (VIF = 25.965 in the quadratic model).
Analysis of Variance
Source DF SS MS F P
Regression 2 2777.0 1388.5 32.57 0.000
Residual Error 12 511.6 42.6
Total 14 3288.7
137
23. Using the final model from problem 22 with H2S = 7.3 and Lactic = 1.85
New
Obs Fit SE Fit 95% CI 95% PI
1 32.36 3.02 (25.78, 38.95) (16.69, 48.04)
Notice the large sample 95% prediction interval is not too much different than the
actual 95% prediction interval (PI) above.
Although the fit in this case is relatively good, the standard error of the estimate is
somewhat large, so there is a fair amount of uncertainty associated with any forecast.
It may be a good idea to collect more data and, perhaps, investigate additional
predictor variables.
138
GtReceit MediaRev StadRev TotRev PlayerCt OpExpens OpIncome
MediaRev 0.304
StadRev 0.587 0.348
TotRev 0.771 0.792 0.753
PlayerCt 0.423 0.450 0.269 0.499
OpExpens 0.636 0.554 0.623 0.766 0.867
OpIncome 0.562 0.672 0.547 0.785 -0.075 0.203
FranValu 0.655 0.780 0.701 0.925 0.397 0.635 0.797
Step 1
Constant 2.928
TotRev 1.96
T-Value 11.94
P-Value 0.000
S 13.7
R-Sq 85.59
R-Sq(adj) 84.99
Results from stepwise program are not surprising given the definitions of the
variables and the strong (and in some cases perfect) multicollinearity.
c. The coefficient of TotRev from the stepwise program is 1.96 and the constant
is relatively small and, in fact, insignificant. Consequently, Franchise Value is,
on average, about twice Total Revenue.
139
Source DF SS MS F P
Regression 1 2101.7 2101.7 72.56 0.000
Residual Error 24 695.2 29.0
Total 25 2796.9
Unusual Observations
The linear relation between Operating expenses and Player costs is fairly strong.
About 75% of the variation in Operating expenses is explained by Player costs.
The actual data for this case is supplied in Appendix A. Students can either be asked to
Respond to the question at the end of the case or they can be assigned to run and analyze the data.
One approach that I have used successfully is to assign one group of students the role of asking
Judy Johnson's questions and another group the responsibility for Ron's answers.
1. What questions do you think Judy will have for Ron? The students always seem
to come up with questions that Ms. Johnson will ask. The key is that Ron should be able
to answer them. Possible issues include:
Are all the predictor variables in the final model required? Is a simpler model
with fewer predictor variables feasible?
Do the estimated regression coefficients in the final model make sense and are
they reliable?
Four observations have large standardized residuals. Is this a cause for concern?
Is the final model a good one and can it be confidently used to forecast the
utility’s bond interest rate at the time of issuance?
140
Is multiple regression the appropriate statistical method to use for this situation?
1. The multiple regression model that includes both unemployment rate and average
monthly temperature is shown below. Temperature is the only good predictor variable.
2. Yes.
Analysis of Variance
Source DF SS MS F P
141
Regression 2 120430208 60215104 48.28 0.000
Residual Error 54 67351116 1247243
Total 56 187781324
The regression is significant. The signs on the coefficients of the independent variables
make sense. The coefficient of each independent variable is significantly different
from 0 (t = –4.6, p value = .000 and t = 4.4, p value = .000, respectively).
Analysis of Variance
Source DF SS MS F P
Regression 3 140771801 46923934 52.90 0.000
Residual Error 53 47009523 886972
Total 56 187781324
Unusual Observations
142
The regression is significant. Each predictor variable is significant. R2 = 75%.
Apart from a couple of large residuals, the residual plots indicate an adequate
model. There is no indication any of the usual regression assumptions have been
violated. A good model has been developed.
1. The regression is significant. The R of 78.1% looks good. The t statistic for each
of the predictor variables is large with a very small p-value. The VIF’s are relatively
small for the three predictors indicating that multicollinearity is not a problem. The
residual plots shown in Figure 7-4 indicate that this model is valid. Dr. Hanke has
developed a good model to forecast ERA.
2. The matrix plot below of ERA versus each of five potential predictor variables does
not show any obvious nonlinear relationships. There does not appear to be any
reason to develop a new model.
143
3. The regression results with WHIP replacing OBA as a predictor variable follow.
The residual plots are very similar to those in Figure 7-4.
Analysis of Variance
Source DF SS MS F P
Regression 3 91.167 30.389 157.48 0.000
Residual Error 134 25.859 0.193
Total 137 117.026
The fit and the adequacy of this model are virtually indistinguishable from the
corresponding model with OBA instead of WHIP as a predictor. The estimated
coefficients of CMD and HR/9 are nearly the same in both models. Both models are
good. The original model with OBA as a predictor has a slightly higher R2 and a
slightly smaller standard error of the estimate. Using these criteria, it is the preferred
model.
144
CASE 7-4: FANTASY BASEBALL (B)
The project may not be doomed to failure. A lot can be learned from investigating the
influence of the various independent variables on WINS. However, the best regression model
does not explain a large percentage of the variation in WINS, R2 = 34%, so the experts have
a point. There will be a lot of uncertainty associated with any forecast of WINS. The stepwise
selection of the best predictor variables and the subsequent full regression output follow.
Step 1 2
Constant 20.531 5.543
RUNS 0.0182
T-Value 3.86
P-Value 0.000
S 3.33 3.17
R-Sq 26.51 33.83
R-Sq(adj) 25.97 32.85
Analysis of Variance
Source DF SS MS F P
Regression 2 695.31 347.66 34.51 0.000
Residual Error 135 1360.17 10.08
Total 137 2055.48
145
CHAPTER 8
1. If not properly accounted for, serial correlation can lead to false inferences under the
usual regression assumptions. Regressions can be judged significant when, in fact,
they are not, coefficient standard errors can be under (or over) estimated so individual
terms in the regression function may be judged significant (or insignificant) when they
are not (or are) and so forth.
2. Serial correlation often arises naturally in time series data. Series, like employment,
whose magnitudes are naturally related to the seasons of the year will be autocorrelated.
Series, like sales, that arise because of a consistently applied mechanism, like advertising
or effort, will be related from one period to the next (serially correlated). In the analysis
of time series data, autocorrelated residuals arise because of a model specification error
or incorrect functional form—the autocorrelation in the series is not properly accounted
for.
4. Durbin-Watson statistic
5. Reject H0 if DW < 1.10. Since 1.0 < 1.10, reject and conclude that the errors are
positively autocorrelated.
6. Reject H0 if DW < 1.55, Do not reject H0 if DW > 1.62. Since 1.6 falls between 1.55
and 1.62, the test is inconclusive.
8. A predictor variable is generated by using the Y variable lagged one or more periods.
146
9. The regression equation is
Fuel = 113 - 8.63 Price - 0.137 Pop
Analysis of Variance
Source DF SS MS F P
Regression 2 223.39 111.69 21.29 0.000
Residual Error 13 68.19 5.25
Total 15 291.58
Using the .05 significance level for a sample size of 16 with 2 predictor variables,
dL = .98. Since DW = .61 < .98, reject H0 and conclude the observations are positively
serially correlated.
Analysis of Variance
Source DF SS MS F P
Regression 3 2.20854E+11 73617995859 15.02 0.000
Residual Error 10 49008480079 4900848008
Total 13 2.69862E+11
147
With n = 14, k =3 and α = .05, DW = 1.14 gives an indeterminate test for serial
correlation.
11. Serial correlation is not a problem. However, it is interesting to see whether the students
realize that collinearity is a likely problem since Customer and Charge are highly correlated.
Correlation matrix:
Analysis of Variance
Source DF SS MS F P
Regression 3 77037 25679 539.30 0.000
Residual Error 24 1143 48
Total 27 78180
148
Analysis of Variance
Source DF SS MS F P
Regression 2 76938 38469 774.66 0.000
Residual Error 25 1241 50
Total 27 78180
The best model, after taking account of the initial multicollinearity, uses the predictor
variables Earnings and Payout (ratio).
Analysis of Variance
Source DF SS MS F P
Regression 2 440912859 220456429 14.33 0.000
Residual Error 25 384584454 15383378
Total 27 825497313
b. With n = 28, k = 2 and α = .01, DW = .29 < dL = 1.04 so there is strong evidence of
positive serial correlation.
b. No. The residual autocorrelation function for the residuals from the straight line fit
indicates significant positive autocorrelation. The independent errors assumption
is not viable.
c. The fitted line plot with the natural logarithms of Passengers as the dependent variable
150
and the residual autocorrelation function follow.
The residual autocorrelation function looks a little better than that in part b,
but there is still significant positive autocorrelation at lag 1.
d. Exponential trend plot for Passengers follows along with residual autocorrelation
function.
151
Still some residual autocorrelation. Errors are not independent.
e. Models in parts c and d are equivalent. If you take the natural logarithms of
fitted exponential growth model you get the fitted model in part c.
f. As we have pointed out, the errors for either of the models in parts c and d are
not independent. Using a model that assumes the errors are independent can
lead to inaccurate forecasts and, in this case, unwarranted precision.
152
Sales = 20.2 + 9.23 Lg2Permits
Forecasts for the 3rd and 4th quarters can be done using several different
approaches. This is best left to the student with a discussion of why they
used a particular method. One method that is to average the past values
of Permits for the 1st and 2nd quarters and use these averages in the model.
This will result in forecasts: 3rd quarter 514; 4th quarter 235.
15.
Quarter Sales S2 S3 S4
1 16.3 0 0 0
2 17.7 1 0 0
3 28.1 0 1 0
4 34.3 0 0 1
153
Sales = 19.3 - 1.43 S2 + 11.2 S3 + 33.3 S4
Analysis of Variance
Source DF SS MS F P
Regression 3 8726.5 2908.8 56.36 0.000
Residual Error 42 2167.6 51.6
Total 45 10894.1
The regression is significant. The model explains 80.1% of the variation in Sales.
There is no lag 1 autocorrelation but a significant residual autocorrelation at lag 4.
154
c. . Calculate the generalized differences and
, and fit the model given in equation (8.5). The result
is with Durbin-Watson statistic = 1.74. In this case, the
estimate of , , is nearly the same as the estimate of in part a.
Here the autocorrelation in the data is not strong enough to have much effect
on the least squares estimate of the slope coefficient.
155
S = 239.721 R-Sq = 53.0% R-Sq(adj) = 50.3%
Analysis of Variance
Source DF SS MS F P
Regression 1 1164598 1164598 20.27 0.000
Residual Error 18 1034389 57466
Total 19 2198987
Analysis of Variance
Source DF SS MS F P
Regression 1 430.0 430.0 4.23 0.054 ← (1) Regression is not
significant
at .01 level
Residual Error 18 1829.0 101.6
Total 19 2259.0
156
Predictor Coef SE Coef T P
Constant -3.141 2.504 -1.25 0.227
Income 0.07632 0.01279 5.97 0.000
War Year 20.165 2.375 8.49 0.000 ← (1) Given Income, War Year
makes
a significant contribution at
the
.01 level.
S = 4.53134 R-Sq = 84.5% R-Sq(adj) = 82.7%
Analysis of Variance
Source DF SS MS F P
Regression 2 1909.94 954.97 46.51 0.000
Residual Error 17 349.06 20.53
Total 19 2259.00
Using all the usual criteria for judging the adequacy of a regression model, this model
is much better than the simple linear regression model in part a.
19. a.
157
The data are clearly seasonal with fourth quarter sales large and sales for the
remaining quarters relatively small. Seasonality is confirmed by the
autocorrelation function with significant autocorrelation at the seasonal
lag 4.
Analysis of Variance
Source DF SS MS F P
Regression 1 4767638 4767638 84.32 0.000
Residual Error 22 1243890 56540
Total 23 6011528
158
Significant lag 1 residual autocorrelation.
Forecasts are not bad but they are below the Value Line estimates for the
last 3 quarters and the difference becomes increasingly larger.
e. Value line estimates for the last 3 quarters of 2003-04 seem increasingly optimistic.
159
The correlations are similar for both the original and natural log transformed data.
Correlations among the potential predictor variables are large implying a
multicollinearity problem. Chicken consumption is most highly correlated with
Income and BeefPrice for both the original and log transformed data. Must be
careful interpreting correlations with time series data since autocorrelation in the
individual series can result in apparent linear association.
Step 1 2
Constant 28.86 37.72
ChickPrice -0.29
T-Value -2.34
P-Value 0.030
S 2.58 2.34
R-Sq 84.98 88.21
R-Sq(adj) 84.27 87.03
c. There is high multicollinearity among the predictor variables so the final model
depends on which non-significant predictor variable is deleted first. If BeefPrice is
deleted, the final model is the one selected by stepwise regression (using a .05 level
for determining significance of individual terms) with significant lag 1 residual
autocorrelation. If Income is deleted first, then the final model involves the three
Price predictor variables as shown below. There is no significant residual
autocorrelation but large VIFs, although the coefficients of the predictor variables
have the right signs. In this data set, Income is essentially a proxy for the three
price variables.
160
S = 2.11241 R-Sq = 90.9% R-Sq(adj) = 89.4%
Analysis of Variance
Source DF SS MS F P
Regression 3 844.44 281.48 63.08 0.000
Residual Error 19 84.78 4.46
Total 22 929.22
Step 1 2
Constant 1.729 2.375
LnChickP -0.445
T-Value -6.06
P-Value 0.000
S 0.0528 0.0321
R-Sq 90.71 96.72
R-Sq(adj) 90.27 96.40
161
Analysis of Variance
Source DF SS MS F P
Regression 2 0.61001 0.30500 295.30 0.000
Residual Error 20 0.02066 0.00103
Total 22 0.63067
Analysis of Variance
Source DF SS MS F P
Regression 2 8.039 4.020 2.72 0.091
Residual Error 19 28.033 1.475
Total 21 36.073
Very little explanatory power in the predictor variables. If the non-significant DiffIncome
is dropped from the model, the resulting regression is significant at the .05 level, R 2 is
virtually unchanged and the standard error of the estimate decreases slightly. The residual
plots look good and there is no evidence of autocorrelation. With the very low R 2, the fitted
162
function is not useful for forecasting the change (difference) in chicken consumption.
Analysis of Variance
Source DF SS MS F P
Regression 1 769.45 769.45 432.71 0.000
Residual Error 20 35.56 1.78
Total 21 805.01
163
Fitted regression function implies this year’s chicken consumption is likely to be
a very good predictor of next year’s chicken consumption. The coefficient on
lagged chicken consumption (LagChickC) is almost 1. The intercept in not significant.
Chicken consumption is essentially a “random walk”—next year’s chicken consumption is
this year’s chicken consumption plus a random amount with mean 0. The residual
plots look good and there is no residual autocorrelation.
We cannot infer the effect of a change in chicken price on chicken consumption with
this model since chicken price does not appear as a predictor variable.
24.
Here the independent error has mean 0 and variance 3σ2. So the first differences for
both and are stationary and X and Y are cointegrated of order 1. The cointegrating
linear combination is: .
2. Would it have been better to eliminate multicollinearity first and then tackle
autocorrelation?
Answer: No. In order to solve the autocorrelation problem, the nature of the data was
changed (first differenced). If multicollinearity were solved first, one or more important
variables may have been eliminated. Autocorrelation must be accounted for first so the
usual regression assumptions apply; then multicollinearity can be tackled.
164
3. How does the small sample size affect the analysis?
Answer: A sample size of 15 is small for a model that uses three independent
variables (ideally, n should be in the neighborhood of 30 or more). A larger sample
size would almost certainly be helpful.
4. Should the regression done on the first differences have been through the origin?
Answer: Perhaps. An intercept can be included in the regression model and then
checked for significance. Ordinarily, regressions with first differenced data does
not require an intercept term.
6. What conclusions can be drawn from a comparison of the Spokane County business
activity index and the GNP?
Answer: The Spokane business activity seems to be extremely stable. It was not
affected by the national recessions of 1970 and 1974. The large peak in 1974 was
caused by Expo 74 (a world fair). It would be inappropriate in this case to expect
the Spokane economy to follow national patterns.
4. Would another type of forecasting model be more effective for forecasting weekly sales?
Answer: Possibly! Jim will investigate Box-Jenkins ARIMA models in Chapter 9.
165
John is correct to be disappointed with the model run with seasonal dummy variables since
the residual autocorrelations have a spike at lag 12. From a forecasting perspective, the
autoregressive model is better. The intercept term allows for a time trend, seasonality is
accounted for by sales lagged 12 months as the predictor variable, R2 is large (91%) and there is
no residual autocorrelation. However, this model does not include predictor variables directly
under John’s control, like price, so he would not be able to determine how a change in price (or
changes in other
operational variables) might affect future sales.
Nonseasonal model:
Analysis of Variance
Source DF SS MS F P
Regression 3 34630 11543 41.62 0.000
Residual Error 80 22187 277
Total 83 56816
The best nonseasonal regression model used the business activity index, number of
bankruptcies filed, and number of building permits to forecast number of clients seen. The
Durbin-Watson test for serial correlation is inconclusive at the .05 level. The residual
autocorrelation function shows some significant autocorrelation around lag 4.
Analysis of Variance
Source DF SS MS F P
Regression 12 39111.7 3259.3 13.07 0.000
Residual Error 71 17704.7 249.4
Total 83 56816.3
The best seasonal model uses Index and 11 seasonal dummy variables to represent
the months Feb through Dec. We retain all the seasonal dummy variables for forecasting
purposes even though some are non-significant. The Durbin-Watson test is inconclusive at the
.05 level. The residual autocorrelations have a just significant spike at lag 6 but are otherwise
non-significant. Forecasts for the first three months of 1993 follow.
Forecast Actual
Jan 1993 179 151
Feb 1993 175 152
Mar 1993 197 199
Forecasts for Jan and Feb 1993 are high compared to actual numbers of clients but
forecast for Mar 1993 is very close to the actual number of new clients
Autoregressive model:
Autoregressive models with number of new clients lagged 1, 4 and 12 months were
167
tried. None of these models proved to be useful for forecasting. The best model had number of
new clients lagged 1 month. The results are displayed below.
Analysis of Variance
Source DF SS MS F P
Regression 1 19035 19035 30.62 0.000
Residual Error 93 57805 622
Total 94 76840
1. The results for the best model are shown below (see also solution to Case 7-2). Each of
the independent variables is significantly different from 0 at the .05 level. The signs of
the coefficients are what we would expect them to be.
Analysis of Variance
Source DF SS MS F P
168
Regression 3 140771801 46923934 52.90 0.000
Residual Error 53 47009523 886972
Total 56 187781324
2. Serial correlation is not a problem. The value of the Durbin-Watson statistic (1.62)
would not reject the null hypothesis of no serial correlation. There are no
significant residual autocorrelations. Restricting attention to integer powers, 2 is the
best choice for the exponential transformation. Allowing other choices for powers,
e.g. 2.4, may improve the fit a bit but is not as “nice” as an integer power.
3. The memo to Mr. DeCoria should use all the usual inferential and descriptive summaries
to defend the model in part 1. A residual analysis should also be included.
2. “Selling” the final regression model to management, including the irascible Jackson
Tilson, ties the statistical exercise in the Alomega case to the real world of business
management. The idea of selling the statistical results to management can be
the focus of team presentations to the class with the instructor playing the role of
Tilson. Working through the presentation of results to the class adds an important
“real world” element to the statistical analysis.
3. As noted in the case, the advertising predictor variables are under the control of
Alomega management. Students can demonstrate the usefulness of this result by
choosing reasonable future values for these advertising variables and generating
forecasts.
However, students must recognize the regression equation does not necessarily
imply a cause and effect relationship between advertising expenditures and sales. In
addition, conditions under which the model was developed may change in the future.
4. All forecasts, including the ones using Julie’s regression equation, assume a future
that is identical to the past except for the identified predictor variables. If her
model is used to generate forecasts for Alomega, she should check the model
accuracy on a regular basis. The errors encountered as the future unfolds should
be compared to those in the data used to generate the model. If significant
changes or trends are observed, the model should be updated to include the most
recent data, along with possibly discarding some of the oldest data. Alternatively,
169
a different approach to the forecasting problem can be sought if the forecasting errors
suggest that the current regression model is inadequate.
1. The positive coefficient on November makes sense because cookie sales are seasonal
sales relatively high each year in November, the month before the Christmas holidays.
2. Jame’s model looks good. Almost 94% of the variation in cookie sales is explained
by the model. The residual analysis indicates the usual regression assumptions are
tenable, including the independence assumption.
3. Forecasts:
Analysis of Variance
Source DF SS MS F P
Regression 1 7.03141E+12 7.03141E+12 118.35 0.000
Residual Error 27 1.60415E+12 59412957997
Total 28 8.63556E+12
This regression model is very reasonable. About 81% of the variation in cookie
sales is explained with the single predictor variable, sales lagged 12 months
170
(Lg12Sales). The usual residual plots look good and there is no significant residual
autocorrelation.
Forecasts:
5. Both models fit the data well. Apart from July 2003, the forecasts generated by the
models are very close to one another. Dummy variable regression explains more of
the variation in cookie sales but the autoregression is simpler. Could make a case for
either model.
1. The regression results along with residual plots and the residual autocorrelation
function follow.
171
S = 155.945 R-Sq = 48.9% R-Sq(adj) = 42.9%
Analysis of Variance
Source DF SS MS F P
Regression 12 2353707 196142 8.07 0.000
Residual Error 101 2456198 24319
Total 113 4809905
Mary has a right to be disappointed. This regression model does not fit well. Even
allowing for seasonality, only the Dec seasonal dummy variable is significant at the
172
.05 level. The residual plots clearly show a poor fit in the middle of the series and
there is a considerable amount of significant residual autocorrelation.
2. Mary might try an autoregression with different choices of lags of total visits
as predictor variable(s). She might try to fit a Box-Jenkins ARIMA model to
be discussed in Chapter 9. Regardless, finding an adequate model for this
time series will be challenging.
173