Regression Analysis
Regression Analysis
Regression Analysis
Correlation only indicates the degree and direction of relationship between two variables.
It does not, necessarily connote a cause-effect relationship. Even when there are grounds
to believe the causal relationship exits, correlation does not tell us which variable is the
cause and which, the effect. For example, the demand for a commodity and its price will
generally be found to be correlated, but the question whether demand depends on price or
vice-versa; will not be answered by correlation.
The dictionary meaning of the ‘regression’ is the act of the returning or going back. The
term ‘regression’ was first used by Francis Galton in 1877 while studying the
relationship between the heights of fathers and sons.
“Regression is the measure of the average relationship between two or more variables in
terms of the original units of data.”
The line of regression is the line, which gives the best estimate to the values of one
variable for any specific values of other variables.
For two variables on regression analysis, there are two regression lines. One line as the
regression of x on y and other is for regression of y on x.
These two regression line show the average relationship between the two variables. The
regression line of y on x gives the most probable value of y for given value of x and the
regression line of x and y gives the most probable values of x for the given value of y.
For perfect correlation, positive or negative i.e. for r= ±, the two lines coincide i.e. we will
find only one straight line. If r=0, i.e. both the variance are independent then the two lines
will cut each other at a right angle. In this case the two lines will be ║to x and y axis.
In equation first x is called the independent variable and y the dependent variable.
Conditional on the x value, the equations gives the variation of y. In other words ,it
means that corresponding to each value of x ,there is whole conditional probability
distribution of y.
Similar discussion holds for the equation second, where y acts as independent variable
and x as dependent variable.
1- The first object is to estimate the dependent variable from known values of
independent variable. This is possible from regression line.
2- The next objective is to obtain a measure of the error involved in using regression
line for estimation.
Correlation and linear regression are not the same. Consider these differences:
• Correlation quantifies the degree to which two variables are related. Correlation
does not find a best-fit line (that is regression). You simply are computing a
correlation coefficient (r) that tells you how much one variable tends to change
when the other one does.
• With correlation you don't have to think about cause and effect. You simply
quantify how well two variables relate to each other. With regression, you do
have to think about cause and effect as the regression line is determined as the
best way to predict Y from X.
• With correlation, it doesn't matter which of the two variables you call "X" and
which you call "Y". You'll get the same correlation coefficient if you swap the
two. With linear regression, the decision of which variable you call "X" and
which you call "Y" matters a lot, as you'll get a different best-fit line if you swap
the two. The line that best predicts Y from X is not the same as the line that
predicts X from Y.
• Correlation is almost always used when you measure both variables. It rarely is
appropriate when one variable is something you experimentally manipulate. With
linear regression, the X variable is often something you experimental manipulate
(time, concentration...) and the Y variable is something you measure.
A large body of techniques for carrying out regression analysis has been developed.
Familiar methods such as linear regression and ordinary least squares regression
are parametric, in that the regression function is defined in terms of a finite number of
unknown parameters that are estimated from the data. Nonparametric regression refers to
techniques that allow the regression function to lie in a specified set of functions, which
may beinfinite-dimensional.
The performance of regression analysis methods in practice depends on the form of the
data-generating process, and how it relates to the regression approach being used. Since
the true form of the data-generating process is not known, regression analysis depends to
some extent on making assumptions about this process. These assumptions are sometimes
(but not always) testable if a large amount of data is available. Regression models for
prediction are often useful even when the assumptions are moderately violated, although
they may not perform optimally. However when carrying out inference using regression
models, especially involving small effects or questions of causality based
on observational data, regression methods must be used cautiously as they can easily give
misleading results.
Underlying assumptions
Classical assumptions for regression analysis include:
The sample must be representative of the population for the inference prediction.
The error is assumed to be a random variable with a mean of zero conditional on
the explanatory variables.
The variables are error-free. If this is not so, modeling may be done using errors-
in-variables model techniques.
The predictors must be linearly independent, i.e. it must not be possible to express
any predictor as a linear combination of the others. SeeMulticollinearity.
The errors are uncorrelated, that is, the variance-covariance matrix of the errors
is diagonal and each non-zero element is the variance of the error.
The variance of the error is constant across observations (homoscedasticity). If
not, weighted least squares or other methods might be used.
These are sufficient (but not all necessary) conditions for the least-squares estimator to
possess desirable properties, in particular, these assumptions imply that the parameter
estimates will be unbiased, consistent, and efficient in the class of linear unbiased
estimators. Many of these assumptions may be relaxed in more advanced treatments.
Regression Coefficient:-
The graph of the estimated regression equation for simple linear regression is a straight
line approximation to the relationship between y and x.
When regression equations obtained directly that is without taking deviation from actual
or assumed mean then the two Normal equations are to be solved simultaneously as
follows;
Remarks:-
1- It may be noted that both the regression coefficient ( x on y means bxy and y on x
means byx ) cannot exceed 1.
Figures are
x= 10, 12,13,17,18
y= 5, 6, 7,9,13
Solution:-
Question 2:- Using the following data obtain regression equation:-
Solution:-
Question 3:-tp12
Question 4:- Find out the likely production corresponding to a rainfall of 40 cms. From
the following data:-
Rail fall (in Cm) Output (in Quintals)
Average 30 50
Standard Deviation 5 10
Solution:-