Contents

List of Figures .................................................................................................................................................................... 2

List of Tables ..................................................................................................................................................................... 3
Chapter 4........................................................................................................................................................................... 4
4.1. Approximation .................................................................................................................................................. 4
4.2. Common Approximation Functions .................................................................................................................. 4
Linear Regression ...................................................................................................................................................... 4
4.3. Criteria to choose appropriate Approximation Function.................................................................................. 4
Using True Errors....................................................................................................................................................... 4
Using Absolute Errors................................................................................................................................................ 5
Using Least Square Errors ......................................................................................................................................... 5
Linear Least-Squares Regression............................................................................................................................... 5
Linearization of Nonlinear Data ................................................................................................................................ 6
Exponential Function ................................................................................................................................................ 6
Power Function ......................................................................................................................................................... 6
Saturation Function................................................................................................................................................... 7
4.4. Least-Squares Polynomial Approximation ........................................................................................................ 9
Quadratic Least-Squares Regression......................................................................................................................... 9
Cubic Least-Squares Regression.............................................................................................................................. 11
4.5. Polynomial Interpolation ................................................................................................................................ 12
Finite Differences Interpolation : Lagrange’s Interpolation.................................................................................... 12
Drawbacks of Lagrange Interpolation ..................................................................................................................... 13
Divided Differences Interpolation : Newton’s Interpolation .................................................................................. 13

Page 1 of 15
 List of Figures
Figure 1: Least Squares Fit V/s Single Polynomial Interpolation .................................................................................... 4
Figure 2: Linear Fit and Error ............................................................................................................................................ 4
Figure 3: Best Fit using true errors................................................................................................................................... 5
Figure 4: Best Fit using absolute errors ........................................................................................................................... 5
Figure 5: Common Functions to linearize Non-linear data (a,d) Exponential (b,e) Power (c,f) Saturation ....... 6
Figure 6: Example 1 : Best-fit curve for Non-linear data ................................................................................................. 7
Figure 7: Example 1 : Best-fit curve for Non-linear data with Saturation Function ....................................................... 7
Figure 8: Example 2 : Best-fit curve for Non-linear data ................................................................................................. 8
Figure 9: Example 2 : Best-fit curve for Non-linear data using Power Function ............................................................ 8
Figure 10: Example 2 : Verification of Best-fit curve for Non-linear data Power Function ........................................... 8
Figure 11: Example 3 : Best-fit curve for Non-linear data using Quadratic Polynomial Function ............................... 11
Figure 12: Example 4 : Comparing Best-fit curve for Non-linear data using Quadratic and Cubic Polynomial
Functions ........................................................................................................................................................................ 12
Figure 13: Lagrange’s Interpolating Polynomials (a) First-degree Polynomial (b) Second-degree Polynomial .......... 12

Page 2 of 15
 List of Tables
Table 1 : Example 1 : Best fit for Non-linear data ............................................................................................................. 8
Table 2 : Example 2 : Best fit for Non-linear data using Power Function ......................................................................... 9
Table 3 : Example : Best fit for Non-linear data using Second Degree Polynomial ........................................................ 10
Table 4 : Example : Best fit for Non-linear data using Third Degree Polynomial ............................................................ 11
Table 5 : Divided Differences Table for five data points ................................................................................................. 14
Table 6 : Divided Differences Table for example problem ............................................................................................. 14

Page 3 of 15
 Chapter 4
4.1. Approximation
Curve-fitting is a procedure where a function is used to fit a given set of data in the “best” possible way without having to match the data
exactly. As a result, while the function does not necessarily yield the exact value at any of the data points, overall it fits the set of data well.
Several types of functions and polynomials of different orders can be used for curve fitting purposes.

Curve-fitting is also known as Approximation.

4.2. Common Approximation Functions

In many other cases, it may be desired to find estimates of values at intermediate points, that is, at the points between the given data points.
This is done through interpolation, a procedure that first determines a polynomial that agrees exactly with the data points, and then uses the
polynomial to find estimates of values at intermediate points. For a small set of data, interpolation over the entire data may be adequately
accomplished using a single polynomial. For large sets of data, however, different polynomials are used in different intervals of the whole data.
This is referred to as spline interpolation.

As mentioned above, a single polynomial may be sufficient for the interpolation of a small set of data. However, when the data has substantial
error, even if the size of data is small, this may no longer be appropriate. Consider Figure 1, which shows a set of seven data points collected
from an experiment. The nature of the data suggests that for the most part, the y values increase with the x values. A single interpolating
polynomial goes through all of the data points, but displays large oscillations in some of the intervals. As a result, the interpolated values near
𝑥 = 1.2, 𝑥 = 2.5 and 𝑥 = 2.9 will be well outside of the range of the original data.

Figure 1: Least Squares Fit V/s Single Polynomial Interpolation Figure 2: Linear Fit and Error

In these types of situations, it makes more sense to find a function that does not necessarily go through all of the data points, but fits the data
well overall. One option, for example, is to fit the “best” straight line into the data. This line is not random and can be generated systematically
via Least-Squares Regression.

Linear Regression
The simplest case of a least-squares regression involves finding a straight line (linear function) in the form:
𝒚 = 𝒂𝟏 𝒙 + 𝒂𝟎
that best fits a set of 𝑛 data points (𝑥1 , 𝑦1 ), (𝑥2 , 𝑦2 ), … , (𝑥𝑛 , 𝑦𝑛 ). Of course, the data first needs to be plotted to see whether the
independent and dependent variables have a somewhat linear relationship. If this is the case, then the coefficients 𝑎1 and 𝑎0 are determined
such that the error associated with the line is minimized. As shown in Figure 2, at each data point (𝑥𝑖 , 𝑦𝑖 ) the error
𝑒𝑖 is defined as the difference between the true value 𝑦𝑖 and the approximate value 𝑎1 𝑥𝑖 + 𝑎0 :
𝒆𝒊 = 𝒚𝒊 − (𝒂𝟏 𝒙𝒊 + 𝒂𝟎 )

4.3. Criteria to choose appropriate Approximation Function

Using True Errors
Different strategies can be considered for determining the best linear fit of a set of 𝑛 data points (𝑥1 , 𝑦1 ), (𝑥2 , 𝑦2 ), … , (𝑥𝑛 , 𝑦𝑛 ). One such
strategy is to minimize the sum of all the individual errors
𝒏 𝒏

𝑬 = ∑ 𝒆𝒊 = ∑ 𝒚𝒊 − (𝒂𝟏 𝒙𝒊 + 𝒂𝟎 )
𝒊=𝟏 𝒊=𝟏

Page 4 of 15
 Figure 3: Best Fit using true errors Figure 4: Best Fit using absolute errors

This criterion, however, does not offer a good measure of how well the line fits the data because, as shown in Figure 3, it allows for positive
and negative individual errors—even very large errors—to cancel out and yield a zero sum (observe that, in Figure 3, 𝑒1 = −𝑒3 and
𝑒2 = −𝑒4 .)

Using Absolute Errors

Another strategy is to minimize the sum of the absolute values of the individual errors. As a result, the individual errors can no longer cancel
out and the total error is always positive.
𝒏 𝒏

𝑬 = ∑|𝒆𝒊 | = ∑|𝒚𝒊 − (𝒂𝟏 𝒙𝒊 + 𝒂𝟎 )|

𝒊=𝟏 𝒊=𝟏
So, we successfully overcame the deficiency in the previous strategy. This criterion, however, suffers from a different drawback; it is not able
to uniquely determine the coefficients that describe the best line fit, because, for a given set of data, several lines can have the same total
error. Figure 4 shows a set of four data points with two line fits that have the same total error (observe that, in Figure 4, |𝑒1 | = |−𝑒2 | and
|𝑒3 | = |−𝑒4 |.)

Using Least Square Errors

The third strategy is to minimize the sum of the squares of the individual errors:
𝒏 𝒏
𝟐
𝑬 = ∑ 𝒆𝒊 = ∑[ 𝒚𝒊 − (𝒂𝟏 𝒙𝒊 + 𝒂𝟎 )]𝟐
𝒊=𝟏 𝒊=𝟏
This criterion uniquely determines the coefficients that describe the best line fit for a given set of data. As in the second strategy, individual
errors cannot cancel each other and the total error is always positive. Also note that small errors get smaller and large errors get larger.

Linear Least-Squares Regression

As decided above, the criterion to find the line 𝑦𝑖 = 𝑎1 𝑥𝑖 + 𝑎0 that best fits the data (𝑥1 , 𝑦1 ), (𝑥2 , 𝑦2 ), … , (𝑥𝑛 , 𝑦𝑛 ) is to determine the
coefficients 𝑎1 and 𝑎0 that minimize
𝒏 𝒏

𝑬 = ∑ 𝒆𝒊 𝟐 = ∑[ 𝒚𝒊 − (𝒂𝟏 𝒙𝒊 + 𝒂𝟎 )]𝟐
𝒊=𝟏 𝒊=𝟏
Noting that 𝐸 is a (nonlinear) function of 𝑎1 and 𝑎0 , it attains its minimum where 𝜕𝐸/𝜕𝑎0 and 𝜕𝐸/𝜕𝑎1 vanish, that is,
𝒏 𝒏
𝝏𝑬
= 𝟎 ⇒ −𝟐 ∑[ 𝒚𝒊 − (𝒂𝟏 𝒙𝒊 + 𝒂𝟎 )] ⇒ ∑[ 𝒚𝒊 − (𝒂𝟏 𝒙𝒊 + 𝒂𝟎 )] = 𝟎
𝝏𝒂𝟎
𝒊=𝟏 𝒊=𝟏
𝒏 𝒏
𝝏𝑬
= 𝟎 ⇒ −𝟐 ∑{𝒙𝒊 [ 𝒚𝒊 − (𝒂𝟏 𝒙𝒊 + 𝒂𝟎 )]} ⇒ ∑{𝒙𝒊 [ 𝒚𝒊 − (𝒂𝟏 𝒙𝒊 + 𝒂𝟎 )]} = 𝟎
𝝏𝒂𝟏
𝒊=𝟏 𝒊=𝟏
Expanding and rearranging the above equations yields a system of two linear equations to be solved for 𝑎1 and 𝑎0 :
𝒏 𝒏

𝒏𝒂𝟎 + (∑ 𝒙𝒊 ) 𝒂𝟏 = ∑ 𝒚𝒊
𝒊=𝟏 𝒊=𝟏
𝒏 𝒏 𝒏

(∑ 𝒙𝒊 ) 𝒂𝟎 + (∑ 𝒙𝒊 𝟐 ) 𝒂𝟏 = ∑ 𝒙𝒊 𝒚𝒊
𝒊=𝟏 𝒊=𝟏 𝒊=𝟏
By Cramer’s rule, the Coefficients of Linear Regression are found as
𝒏(∑𝒏𝒊=𝟏 𝒙𝒊 𝒚𝒊 ) − (∑𝒏𝒊=𝟏 𝒙𝒊 )(∑𝒏𝒊=𝟏 𝒚𝒊 )
𝒂𝟏 =
𝒏(∑𝒏𝒊=𝟏 𝒙𝒊 𝟐 ) − (∑𝒏𝒊=𝟏 𝒙𝒊 )𝟐
𝒏(∑𝒊=𝟏 𝒙𝒊 𝟐 )(∑𝒏𝒊=𝟏 𝒚𝒊 ) − (∑𝒏𝒊=𝟏 𝒙𝒊 )(∑𝒏𝒊=𝟏 𝒙𝒊 𝒚𝒊 )
𝒏
𝒂𝟎 =
𝒏(∑𝒏𝒊=𝟏 𝒙𝒊 𝟐 ) − (∑𝒏𝒊=𝟏 𝒙𝒊 )𝟐
Example : Using least-squares regression, find a straight line that best fits the following data : (0.2 , 8.0), (0.4 , 8.4), (0.6 , 8.8), (0.8 , 8.6),
(1.0 , 8.5), (1.2 , 8.7).

Page 5 of 15
Solution : First, let us calculate all the sums :
𝟔 𝟔

∑ 𝒙𝒊 = ∑( 𝟎. 𝟐 + 𝟎. 𝟒 + 𝟎. 𝟔 + 𝟎. 𝟖 + 𝟏. 𝟎 + 𝟏. 𝟐) = 𝟒. 𝟐 ∑ 𝒚𝒊 = ∑( 𝟖. 𝟎 + 𝟖. 𝟒 + 𝟖. 𝟖 + 𝟖. 𝟔 + 𝟖. 𝟓 + 𝟖. 𝟕) = 𝟓𝟏. 𝟎
𝒊=𝟏 𝒊=𝟏
𝟔

∑ 𝒙𝒊 𝟐 = ∑( 𝟎. 𝟐𝟐 + 𝟎. 𝟒𝟐 + 𝟎. 𝟔𝟐 + 𝟎. 𝟖𝟐 + 𝟏. 𝟎𝟐 + 𝟏. 𝟐𝟐 ) = 𝟑. 𝟔𝟒
𝒊=𝟏
𝟔

∑ 𝒙𝒊 𝒚𝒊 = ∑[(𝟎. 𝟐)(𝟖. 𝟎) + (𝟎. 𝟒)(𝟖. 𝟒) + (𝟎. 𝟔)(𝟖. 𝟖) + (𝟎. 𝟖)(𝟖. 𝟔) + (𝟏. 𝟎)(𝟖. 𝟓) + (𝟏. 𝟐)(𝟖. 𝟕)] = 𝟑𝟔. 𝟎𝟔
𝒊=𝟏
Now,
(𝟔)(𝟑𝟔. 𝟎𝟔) − (𝟒. 𝟐)(𝟓𝟏) (𝟔)(𝟑. 𝟔𝟒)(𝟓𝟏) − (𝟒. 𝟐)(𝟑𝟔. 𝟎𝟔)
𝒂𝟏 = = 𝟎. 𝟓𝟏𝟒𝟑 𝒂𝟎 = = 𝟖. 𝟏𝟒
(𝟔)(𝟑. 𝟔𝟒) − (𝟒. 𝟐)𝟐 (𝟔)(𝟑. 𝟔𝟒) − (𝟒. 𝟐)𝟐

Therefore, the line that best fits the data is described by 𝑦 = 0.5143𝑥 + 8.14

Linearization of Nonlinear Data

If the relationship between the independent and dependent variables is non-linear, curve-fitting techniques other than linear regression must
be used. To fit non-linear data, usually we follow one of the two methods : (a) Conversion of non-linear data into equivalent auxiliary linear
data, and (b) Using Polynomial Regression. The second approach, Polynomial Regression, will be explained later. Let us understand the former
approach first.

In general, non-linear data fits into one of the three curve-fitting functions, as follows.

Exponential Function
One is the exponential function :
𝒚 = 𝒂𝒆𝒃𝒙 (𝑎, 𝑏 𝑎𝑟𝑒 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡𝑠)
Because differentiation of the exponential function returns a constant multiple of the exponential function, this technique applies to situations
where the rate of change of a quantity is directly proportional to the quantity itself; for instance, radioactive decay.

Conversion into linear form is made by taking the natural logarithm on both sides of equation to obtain
𝐥𝐧 𝒚 = 𝒃𝒙 + 𝐥𝐧 𝒂
Therefore, the plot of ln 𝑦 versus 𝑥 is a straight line with slope 𝑏 and intercept ln 𝑎 (see Figure 5 (a) and (d).)

Figure 5: Common Functions to linearize Non-linear data (a,d) Exponential (b,e) Power (c,f) Saturation

Power Function
Another strategy to fit nonlinear function is the power function

Page 6 of 15
 𝒚 = 𝒂𝒙𝒃 (𝑎, 𝑏 𝑎𝑟𝑒 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡𝑠)
Linearization is achieved by taking the common logarithm (base 10) on both sides of equation:
𝐥𝐨𝐠 𝒚 = 𝒃 𝐥𝐨𝐠 𝒙 + 𝐥𝐨𝐠 𝒂
so that the plot of log 𝑦 versus log 𝑥 is a straight line with slope 𝑏 and intercept log 𝑎 (see Figure 5 (b) and (e).)

Saturation Function
The saturation function is in the form
𝒙
𝒚= (𝑎, 𝑏 𝑎𝑟𝑒 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡𝑠)
𝒂𝒙 + 𝒃
Inverting both sides of equation yields
𝟏 𝟏
= 𝒃( )+𝒂
𝒚 𝒙
so that the plot of 1⁄𝑦 versus 1⁄𝑥 is a straight line with slope 𝑏 and intercept 𝑎 (see Figure 5 (c) and (f).)

Example : Find the best fit curve for the following data : (10 , 1.9), (20 , 3.0), (30 , 3.2), (40 , 3.9), (50 , 3.7), (60 , 4.2) , (70 , 4.1), (80 , 4.4),
(90 , 4.5), (100 , 4.4).

Solution : To find the best fit it is best to plot the given data. It is shown in Figure 6.

Figure 6: Example 1 : Best-fit curve for Non-linear data Figure 7: Example 1 : Best-fit curve for Non-linear data with
Saturation Function

Looking at the graph intuition tells us that the function that best fits this data should be Saturation Function. But, let us not go through our
intuition, but rather try to fit this data in all three possible functions (viz. exponential, power, and saturation). To do that we should find
equivalent auxiliary data for given data (𝑥𝑖 , 𝑦𝑖 ) , 1 ≤ 𝑖 ≤ 10. Let us name these auxiliary data as follows: Exponential (𝑥𝑒𝑖 , 𝑦𝑒𝑖 ) =
(ln 𝑥𝑖 , ln 𝑦𝑖 ), Power (𝑥𝑝𝑖 , 𝑦𝑝𝑖 ) = (log 𝑥𝑖 , log 𝑦𝑖 ), and Saturation (𝑥𝑠𝑖 , 𝑦𝑠𝑖 ) = (1⁄𝑥𝑖 , 1⁄𝑦𝑖 ). After producing auxiliary data it is easy to
find best fit linear curve for these auxiliary data:
Exponential : 𝒚𝒆𝒊 = 𝒂𝒆𝟏 𝒙𝒆𝒊 + 𝒂𝒆𝟎 Power : 𝒚𝒑𝒊 = 𝒂𝒑𝟏 𝒙𝒑𝒊 + 𝒂𝒑𝟎 Saturation : 𝒚𝒔𝒊 = 𝒂𝒔𝟏 𝒙𝒔𝒊 + 𝒂𝒔𝟎
Finally, to find the most appropriate function we calculate the errors, and pick up the function which gives least error :
Exponential : Power : Saturation :
𝟏𝟎 𝟏𝟎 𝟏𝟎 𝟏𝟎 𝟏𝟎 𝟏𝟎
𝟐
𝑬𝒆 = ∑ 𝒆𝒆𝒊 𝟐 = ∑[ 𝒚𝒆𝒊 − (𝒂𝒆𝟏 𝒙𝒆𝒊 + 𝒂𝒆𝟎 )]𝟐 𝑬𝒑 = ∑ 𝒆𝒑𝒊 𝟐 = ∑[ 𝒚𝒑𝒊 − (𝒂𝒑𝟏 𝒙𝒑𝒊 + 𝒂𝒑𝟎 )] 𝑬𝒔 = ∑ 𝒆𝒔𝒊 𝟐 = ∑[ 𝒚𝒔𝒊 − (𝒂𝒔𝟏 𝒙𝒔𝒊 + 𝒂𝒔𝟎 )]𝟐
𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏

All these calculations are shown in Table 1. The remaining six quantities which are not shown in Table 1 are as follows:

(𝟏𝟎)(𝟓𝟎. 𝟖𝟏𝟕𝟕) − (𝟑𝟖. 𝟏𝟑𝟎𝟑)(𝟏𝟐. 𝟖𝟖𝟔𝟑) (𝟏𝟎)(𝟏𝟓𝟎. 𝟐𝟐𝟕𝟔)(𝟏𝟐. 𝟖𝟖𝟔𝟑) − (𝟑𝟖. 𝟏𝟑𝟎𝟑)(𝟓𝟎. 𝟖𝟏𝟕𝟕)
𝒂𝒆𝟏 = = 𝟎. 𝟑𝟒𝟕𝟖 𝒂𝒆𝟎 = = 𝟑𝟔𝟎. 𝟐𝟒𝟑𝟑
(𝟏𝟎)(𝟏𝟓𝟎. 𝟐𝟐𝟕𝟔) − (𝟑𝟖. 𝟏𝟑𝟎𝟑)𝟐 (𝟏𝟎)(𝟏𝟓𝟎. 𝟐𝟐𝟕𝟔) − (𝟑𝟖. 𝟏𝟑𝟎𝟑)𝟐
(𝟏𝟎)(𝟗. 𝟓𝟖𝟒𝟖) − (𝟏𝟔. 𝟓𝟓𝟗𝟖)(𝟓. 𝟓𝟗𝟔𝟒) (𝟏𝟎)(𝟐𝟖. 𝟑𝟑𝟒𝟕)(𝟓. 𝟓𝟗𝟔𝟒) − (𝟏𝟔. 𝟓𝟓𝟗𝟖)(𝟗. 𝟓𝟖𝟒𝟖)
𝒂𝒑𝟏 = = 𝟎. 𝟑𝟒𝟕𝟖 𝒂𝒑𝟎 = = 𝟏𝟓𝟔. 𝟒𝟓𝟏𝟕
(𝟏𝟎)(𝟐𝟖. 𝟑𝟑𝟒𝟕) − (𝟏𝟔. 𝟓𝟓𝟗𝟖)𝟐 (𝟏𝟎)(𝟐𝟖. 𝟑𝟑𝟒𝟕) − (𝟏𝟔. 𝟓𝟓𝟗𝟖)𝟐
(𝟏𝟎)(𝟎. 𝟏𝟎𝟔𝟔) − (𝟎. 𝟐𝟗𝟐𝟗)(𝟐. 𝟖𝟓𝟕𝟔) (𝟏𝟎)(𝟎. 𝟎𝟏𝟓𝟓)(𝟐. 𝟖𝟓𝟕𝟔) − (𝟎. 𝟐𝟗𝟐𝟗)(𝟎. 𝟏𝟎𝟔𝟔)
𝒂𝒔𝟏 = = 𝟑. 𝟑𝟎𝟓𝟐 𝒂𝒔𝟎 = = 𝟓. 𝟗𝟒𝟗𝟕
(𝟏𝟎)(𝟎. 𝟎𝟏𝟓𝟓) − (𝟎. 𝟐𝟗𝟐𝟗)𝟐 (𝟏𝟎)(𝟎. 𝟎𝟏𝟓𝟓) − (𝟎. 𝟐𝟗𝟐𝟗)𝟐

Hence, as is evident from the calculations, the given data fits best to a Saturation Function as follows:

𝟏 𝟏 𝒙
= 𝟑. 𝟑𝟎𝟓𝟐 + 𝟓. 𝟗𝟒𝟗𝟕 or 𝒚=
𝒚 𝒙 𝟓.𝟗𝟒𝟗𝟕𝒙+𝟑.𝟑𝟎𝟓𝟐

This can also be verified from Figure 7.

Page 7 of 15
 Table 1 : Example 1 : Best fit for Non-linear data

i 𝒙𝒊 𝒚𝒊 𝒙𝒆𝒊 𝒚𝒆𝒊 𝒙𝒑𝒊 𝒚𝒑𝒊 𝒙𝒔𝒊 𝒚𝒔𝒊 𝒙𝒆𝒊 𝟐 𝒙𝒆𝒊 𝒚𝒆𝒊 𝒙𝒑𝒊 𝟐 𝒙𝒑𝒊 𝒚𝒑𝒊 𝒙𝒔𝒊 𝟐 𝒙𝒔𝒊 𝒚𝒔𝒊 𝒆𝒆𝒊 𝟐 𝒆𝒑𝒊 𝟐 𝒆𝒔𝒊 𝟐
1 10 1.9 2.3026 0.6419 1.0000 0.2788 0.1000 0.5263 5.3019 1.4779 1.0000 0.2788 0.0100 0.0526 128984.6 24195.4 57.6
2 20 3.0 2.9957 1.0986 1.3010 0.4771 0.0500 0.3333 8.9744 3.2911 1.6927 0.6207 0.0025 0.0167 128368.3 24123.2 49.0
3 30 3.2 3.4012 1.1632 1.4771 0.5051 0.0333 0.3125 11.5681 3.9561 2.1819 0.7462 0.0011 0.0104 128326.1 24071.6 48.3
4 40 3.9 3.6889 1.3610 1.6021 0.5911 0.0250 0.2564 13.6078 5.0205 2.5666 0.9469 0.0006 0.0064 127896.6 24042.1 45.9
5 50 3.7 3.9120 1.3083 1.6990 0.5682 0.0200 0.2703 15.3039 5.1182 2.8865 0.9654 0.0004 0.0054 128095.3 24009.6 46.6
6 60 4.2 4.0943 1.4351 1.7782 0.6232 0.0167 0.2381 16.7637 5.8757 3.1618 1.1082 0.0003 0.0040 127782.9 23991.0 45.2
7 70 4.1 4.2485 1.4110 1.8451 0.6128 0.0143 0.2439 18.0497 5.9946 3.4044 1.1306 0.0002 0.0035 127892.8 23969.1 45.4
8 80 4.4 4.3820 1.4816 1.9031 0.6435 0.0125 0.2273 19.2022 6.4924 3.6218 1.2245 0.0002 0.0028 127711.5 23954.5 44.7
9 90 4.5 4.4998 1.5041 1.9542 0.6532 0.0111 0.2222 20.2483 6.7681 3.8191 1.2765 0.0001 0.0025 127669.3 23939.7 44.5
10 100 4.4 4.6052 1.4816 2.0000 0.6435 0.0100 0.2273 21.2076 6.8230 4.0000 1.2869 0.0001 0.0023 127767.0 23924.5 44.8
∑→ 38.1303 12.8863 16.5598 5.5964 0.2929 2.8576 150.2276 50.8177 28.3347 9.5848 0.0155 0.1066 1280494.3 240220.8 472.0

Example : Find a Power Function that best fits the following data : (10 , 1.9), (20 , 3.0), (30 , 3.2), (40 , 3.9), (50 , 3.7), (60 , 4.2) , (70 , 4.1),
(80 , 4.4), (90 , 4.5), (100 , 4.4).

Solution : The data is plotted in graph of Figure 8.

Figure 8: Example 2 : Best-fit curve for Non-linear data

Figure 9: Example 2 : Best-fit curve for Non-linear data using Figure 10: Example 2 : Verification of Best-fit curve for Non-linear
Power Function data Power Function

Auxiliary data as Power Function is calculated as follows: Power (𝑥𝑝𝑖 , 𝑦𝑝𝑖 ) = (log 𝑥𝑖 , log 𝑦𝑖 ). Using this auxiliary data the best fit linear
curve is found using Table 2. The linear coefficients of auxiliary equation are calculated as:
(𝟏𝟎)(𝟑. 𝟎𝟐𝟒𝟏) − (−𝟑. 𝟒𝟒𝟎𝟐)(−𝟑. 𝟓𝟒𝟗𝟏) (𝟏𝟎)(𝟐. 𝟎𝟗𝟓𝟔)(−𝟑. 𝟓𝟒𝟗𝟏) − (−𝟑. 𝟒𝟒𝟎𝟐)(𝟑. 𝟎𝟐𝟒𝟏)
𝒂𝒑𝟏 = = 𝟏. 𝟗𝟕𝟔𝟗 𝒂𝒑𝟎 = = 𝟎. 𝟑𝟐𝟓𝟐
(𝟏𝟎)(𝟐. 𝟎𝟗𝟓𝟔) − (𝟓. 𝟓)𝟐 (𝟏𝟎)(𝟐. 𝟎𝟗𝟓𝟔) − (𝟓. 𝟓)𝟐
Auxiliary Power Function : 𝒚𝒑𝒊 = 𝒂𝒑𝟏 𝒙𝒑𝒊 + 𝒂𝒑𝟎 = 𝟏. 𝟗𝟕𝟔𝟗𝒙𝒑𝒊 + 𝟎. 𝟑𝟐𝟓𝟐 Power Function : 𝒚 = 𝒆𝒂𝒑𝟎 𝒙𝒂𝒑𝟏 = 𝟐. 𝟏𝟏𝟒𝟓𝒙𝟏.𝟗𝟕𝟔𝟗

The calculations can be verified from Figure 9 and Figure 10.

Page 8 of 15
 Table 2 : Example 2 : Best fit for Non-linear data using Power Function

i 𝒙𝒊 𝒚𝒊 𝒙𝒑𝒊 𝒚𝒑𝒊 𝒙𝒑𝒊 𝟐 𝒙𝒑𝒊 𝒚𝒑𝒊 𝒆𝒑𝒊 𝟐

1 0.1 0.02 -1.0000 -1.6990 1.0000 1.6990 87.8
2 0.2 0.10 -0.6990 -1.0000 0.4886 0.6990 68.7
3 0.3 0.20 -0.5229 -0.6990 0.2734 0.3655 62.0
4 0.4 0.35 -0.3979 -0.4559 0.1584 0.1814 56.5
5 0.5 0.56 -0.3010 -0.2518 0.0906 0.0758 52.0
6 0.6 0.75 -0.2218 -0.1249 0.0492 0.0277 49.5
7 0.7 1.04 -0.1549 0.0170 0.0240 -0.0026 46.6
8 0.8 1.30 -0.0969 0.1139 0.0094 -0.0110 44.8
9 0.9 1.70 -0.0458 0.2304 0.0021 -0.0105 42.4
10 1 2.09 0.0000 0.3201 0.0000 0.0000 40.7
∑ → 5.5 8.11 -3.4402 -3.5491 2.0956 3.0241 551.1

4.4. Least-Squares Polynomial Approximation

In the previous section, we learned that a curve can fit into nonlinear data by transforming the data into a form that can be handled by Linear
Regression. Another method is to fit polynomials of different orders to the data by means of Polynomial Regression.

The Linear Least-Squares Regression of previous section can be extended to fit a set of 𝑛 data points (𝑥1 , 𝑦1 ), (𝑥2 , 𝑦2 ), … , (𝑥𝑛 , 𝑦𝑛 ) with an
𝑚𝑡ℎ − 𝑑𝑒𝑔𝑟𝑒𝑒 polynomial in the form
𝒚 = 𝒂𝒎 𝒙𝒎 + 𝒂𝒎−𝟏 𝒙𝒎−𝟏 + ⋯ + 𝒂𝟐 𝒙𝟐 + 𝒂𝟏 𝒙 + 𝒂𝟎
with a total error
𝒏 𝒏
𝟐
𝑬 = ∑ 𝒆𝒊 = ∑[ 𝒚𝒊 − ( 𝒂𝒎 𝒙𝒊 𝒎 + 𝒂𝒎−𝟏 𝒙𝒊 𝒎−𝟏 + ⋯ + 𝒂𝟐 𝒙𝒊 𝟐 + 𝒂𝟏 𝒙𝒊 + 𝒂𝟎 ) ]
𝟐

𝒊=𝟏 𝒊=𝟏
The coefficients 𝑎𝑚 , 𝑎𝑚−1 , … , 𝑎2 , 𝑎1 , 𝑎0 are to be determined such that 𝐸 is minimized. A necessary condition for 𝐸 to attain a minimum is
that its partial derivative with respect to each of these coefficients vanishes, that is
𝒏
𝝏𝑬
= 𝟎 ⇒ −𝟐 ∑[𝒚𝒊 − ( 𝒂𝒎 𝒙𝒊 𝒎 + 𝒂𝒎−𝟏 𝒙𝒊 𝒎−𝟏 + ⋯ + 𝒂𝟐 𝒙𝒊 𝟐 + 𝒂𝟏 𝒙𝒊 + 𝒂𝟎 )] = 𝟎
𝝏𝒂𝟎
𝒊=𝟏
𝒏
𝝏𝑬
= 𝟎 ⇒ −𝟐 ∑{𝒙𝒊 [𝒚𝒊 − ( 𝒂𝒎 𝒙𝒊 𝒎 + 𝒂𝒎−𝟏 𝒙𝒊 𝒎−𝟏 + ⋯ + 𝒂𝟐 𝒙𝒊 𝟐 + 𝒂𝟏 𝒙𝒊 + 𝒂𝟎 )]} = 𝟎
𝝏𝒂𝟏
𝒊=𝟏
𝒏
𝝏𝑬
= 𝟎 ⇒ −𝟐 ∑{𝒙𝒊 𝟐 [𝒚𝒊 − ( 𝒂𝒎 𝒙𝒊 𝒎 + 𝒂𝒎−𝟏 𝒙𝒊 𝒎−𝟏 + ⋯ + 𝒂𝟐 𝒙𝒊 𝟐 + 𝒂𝟏 𝒙𝒊 + 𝒂𝟎 )]} = 𝟎
𝝏𝒂𝟐
𝒊=𝟏
⋮ ⇒ ⋮ = ⋮
𝒏
𝝏𝑬
= 𝟎 ⇒ −𝟐 ∑{𝒙𝒊 𝒎 [𝒚𝒊 − ( 𝒂𝒎 𝒙𝒊 𝒎 + 𝒂𝒎−𝟏 𝒙𝒊 𝒎−𝟏 + ⋯ + 𝒂𝟐 𝒙𝒊 𝟐 + 𝒂𝟏 𝒙𝒊 + 𝒂𝟎 )]} = 𝟎
𝝏𝒂𝒎
𝒊=𝟏
Manipulation of these equations yields a system of 𝑚 + 1 linear equations to be solved for 𝑎𝑚 , 𝑎𝑚−1 , … , 𝑎2 , 𝑎1 , 𝑎0 :
𝒏 𝒏 𝒏 𝒏 𝒏

(∑ 𝒙𝒊 𝟎 ) 𝒂𝟎 + (∑ 𝒙𝒊 𝟏 ) 𝒂𝟏 + (∑ 𝒙𝒊 𝟐 ) 𝒂𝟐 + ⋯ + (∑ 𝒙𝒊 𝒎+𝟎 ) 𝒂𝒎 = ∑ 𝒙𝒊 𝟎 𝒚𝒊
𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏
𝒏 𝒏 𝒏 𝒏 𝒏

(∑ 𝒙𝒊 𝟏 ) 𝒂𝟎 + (∑ 𝒙𝒊 𝟐 ) 𝒂𝟏 + (∑ 𝒙𝒊 𝟑 ) 𝒂𝟐 + ⋯ + (∑ 𝒙𝒊 𝒎+𝟏 ) 𝒂𝒎 = ∑ 𝒙𝒊 𝟏 𝒚𝒊
𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏
𝒏 𝒏 𝒏 𝒏 𝒏

(∑ 𝒙𝒊 𝟐 ) 𝒂𝟎 + (∑ 𝒙𝒊 𝟑 ) 𝒂𝟏 + (∑ 𝒙𝒊 𝟑 ) 𝒂𝟐 + ⋯ + (∑ 𝒙𝒊 𝒎+𝟐 ) 𝒂𝒎 = ∑ 𝒙𝒊 𝟐 𝒚𝒊
𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏
⋯ = ⋯
𝒏 𝒏 𝒏 𝒏 𝒏

(∑ 𝒙𝒊 𝒎 ) 𝒂𝟎 + (∑ 𝒙𝒊 𝒎+𝟏 ) 𝒂𝟏 + (∑ 𝒙𝒊 𝒎+𝟐 ) 𝒂𝟐 + ⋯ + (∑ 𝒙𝒊 𝒎+𝒎 ) 𝒂𝒎 = ∑ 𝒙𝒊 𝒎 𝒚𝒊

𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏

Quadratic Least-Squares Regression

The objective is to fit a set of 𝑛 data points (𝑥1 , 𝑦1 ), (𝑥2 , 𝑦2 ), … , (𝑥𝑛 , 𝑦𝑛 ) with an 𝑠𝑒𝑐𝑜𝑛𝑑 − 𝑑𝑒𝑔𝑟𝑒𝑒 polynomial in the form
𝒚 = 𝒂𝟐 𝒙𝟐 + 𝒂𝟏 𝒙 + 𝒂𝟎
Such that the total error
Page 9 of 15
 𝒏 𝒏
𝟐
𝑬 = ∑ 𝒆𝒊 𝟐 = ∑[ 𝒚𝒊 − ( 𝒂𝟐 𝒙𝒊 𝟐 + 𝒂𝟏 𝒙𝒊 + 𝒂𝟎 ) ]
𝒊=𝟏 𝒊=𝟏
is minimized. Following the general procedure outlined above, the coefficients 𝑎2 , 𝑎1 , 𝑎0 are determined by solving a system of three linear
equations:
𝒏 𝒏 𝒏 𝒏

(∑ 𝒙𝒊 𝟎 ) 𝒂𝟎 + (∑ 𝒙𝒊 𝟏 ) 𝒂𝟏 + (∑ 𝒙𝒊 𝟐 ) 𝒂𝟐 = ∑ 𝒙𝒊 𝟎 𝒚𝒊
𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏
𝒏 𝒏 𝒏 𝒏

(∑ 𝒙𝒊 𝟏 ) 𝒂𝟎 + (∑ 𝒙𝒊 𝟐 ) 𝒂𝟏 + (∑ 𝒙𝒊 𝟑 ) 𝒂𝟐 = ∑ 𝒙𝒊 𝟏 𝒚𝒊
𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏
𝒏 𝒏 𝒏 𝒏

(∑ 𝒙𝒊 𝟐 ) 𝒂𝟎 + (∑ 𝒙𝒊 𝟑 ) 𝒂𝟏 + (∑ 𝒙𝒊 𝟒 ) 𝒂𝟐 = ∑ 𝒙𝒊 𝟐 𝒚𝒊
𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏
Example : Using Quadratic Least-Squares Regression, find the second-degree polynomial that best fits the following data : (0 , 2.9), (0.4 , 3.1),
(0.8 , 3.56), (1.2 , 4.6) , (1.6 , 6.7).
Solution : The aim is to find the coefficients 𝑎2 , 𝑎1 , 𝑎0 of the equation 𝑦 = 𝑎2 𝑥 2 + 𝑎1 𝑥 + 𝑎0 by solving the three equations:
𝟓 𝟓 𝟓 𝟓

(∑ 𝒙𝒊 𝟎) 𝒂 + (∑ 𝒙𝒊 𝟏) 𝒂 + (∑ 𝒙𝒊 𝟐) 𝒂 = ∑ 𝒙𝒊 𝟎 𝒚𝒊
𝟎 𝟏 𝟐
𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏
𝟓 𝟓 𝟓 𝟓
𝟏) 𝒂 𝟐) 𝒂 𝟑) 𝒂
(∑ 𝒙𝒊 𝟎 + (∑ 𝒙𝒊 𝟏 + (∑ 𝒙𝒊 𝟐 = ∑ 𝒙𝒊 𝟏 𝒚𝒊
𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏
𝟓 𝟓 𝟓 𝟓
𝟐) 𝒂 𝟑) 𝒂 𝟒) 𝒂
(∑ 𝒙𝒊 𝟎 + (∑ 𝒙𝒊 𝟏 + (∑ 𝒙𝒊 𝟐 = ∑ 𝒙𝒊 𝟐 𝒚𝒊
𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏

To do so we prepare Table 3:
Table 3 : Example : Best fit for Non-linear data using Second Degree Polynomial

i 𝒙𝒊 𝒚𝒊 𝒙𝒊 𝟐 𝒙𝒊 𝟑 𝒙𝒊 𝟒 𝒙𝒊 𝒚𝒊 𝒙𝒊 𝟐 𝒚𝒊
1 0.0 2.90 0.0000 0.0000 0.0000 0.0000 0.0000
2 0.4 3.10 0.1600 0.0640 0.0256 1.2400 0.4960
3 0.8 3.56 0.6400 0.5120 0.4096 2.8480 2.2784
4 1.2 4.60 1.4400 1.7280 2.0736 5.5200 6.6240
5 1.6 6.70 2.5600 4.0960 6.5536 10.7200 17.1520
∑ → 4.0 20.86 4.8000 6.4000 9.0624 20.3280 26.5504
Hence, the three equations to be solved are:
𝟓𝒂𝟎 + 𝟒𝒂𝟏 + 𝟒. 𝟖𝒂𝟐 = 𝟐𝟎. 𝟖𝟔
𝟒𝒂𝟎 + 𝟒. 𝟖𝒂𝟏 + 𝟔. 𝟒𝒂𝟐 = 𝟐𝟎. 𝟑𝟐𝟖
𝟒. 𝟖𝒂𝟎 + 𝟔. 𝟒𝒂𝟏 + 𝟗. 𝟎𝟔𝟐𝟒𝒂𝟐 = 𝟐𝟔. 𝟓𝟓𝟎𝟒
Solution of these equations yield 𝑎2 = 1.9554, 𝑎1 = −0.8536, 𝑎0 = 2.9777. Hence, the quadratic function which best fits the given set of
data is 𝑦 = 1.9554𝑥 2 − 0.8536𝑥 + 2.9777. This can be verified from Figure 11

Page 10 of 15
 Figure 11: Example 3 : Best-fit curve for Non-linear data using Quadratic Polynomial Function

Cubic Least-Squares Regression

The objective is to fit a set of 𝑛 data points (𝑥1 , 𝑦1 ), (𝑥2 , 𝑦2 ), … , (𝑥𝑛 , 𝑦𝑛 ) with an 𝑡ℎ𝑖𝑟𝑑 − 𝑑𝑒𝑔𝑟𝑒𝑒 polynomial in the form
𝒚 = 𝒂𝟑 𝒙𝟑 + 𝒂𝟐 𝒙𝟐 + 𝒂𝟏 𝒙 + 𝒂𝟎
Such that the total error
𝒏 𝒏
𝟐
𝑬 = ∑ 𝒆𝒊 = ∑[ 𝒚𝒊 − ( 𝒂𝟑 𝒙𝟑 + 𝒂𝟐 𝒙𝒊 𝟐 + 𝒂𝟏 𝒙𝒊 + 𝒂𝟎 ) ]
𝟐

𝒊=𝟏 𝒊=𝟏
is minimized. Following the general procedure outlined above, the coefficients 𝑎3 , 𝑎2 , 𝑎1 , 𝑎0 are determined by solving a system of four linear
equations:
𝒏 𝒏 𝒏 𝒏 𝒏
𝟎) 𝒂 𝟏) 𝒂 𝟐) 𝒂 𝟑) 𝒂
(∑ 𝒙𝒊 𝟎 + (∑ 𝒙𝒊 𝟏 + (∑ 𝒙𝒊 𝟐 + (∑ 𝒙𝒊 𝟑 = ∑ 𝒙𝒊 𝟎 𝒚𝒊
𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏
𝒏 𝒏 𝒏 𝒏 𝒏

(∑ 𝒙𝒊 𝟏 ) 𝒂𝟎 + (∑ 𝒙𝒊 𝟐 ) 𝒂𝟏 + (∑ 𝒙𝒊 𝟑 ) 𝒂𝟐 + (∑ 𝒙𝒊 𝟒 ) 𝒂𝟑 = ∑ 𝒙𝒊 𝟏 𝒚𝒊
𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏
𝒏 𝒏 𝒏 𝒏 𝒏

(∑ 𝒙𝒊 𝟐 ) 𝒂𝟎 + (∑ 𝒙𝒊 𝟑 ) 𝒂𝟏 + (∑ 𝒙𝒊 𝟒 ) 𝒂𝟐 + (∑ 𝒙𝒊 𝟓 ) 𝒂𝟑 = ∑ 𝒙𝒊 𝟐 𝒚𝒊
𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏
𝒏 𝒏 𝒏 𝒏 𝒏

(∑ 𝒙𝒊 𝟑 ) 𝒂𝟎 + (∑ 𝒙𝒊 𝟒 ) 𝒂𝟏 + (∑ 𝒙𝒊 𝟓 ) 𝒂𝟐 + (∑ 𝒙𝒊 𝟔 ) 𝒂𝟑 = ∑ 𝒙𝒊 𝟑 𝒚𝒊
𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏
Example : Using Cubic Least-Squares Regression, find the third-degree polynomial that best fits the following data : (0 , 2.9), (0.4 , 3.1),
(0.8 , 3.56), (1.2 , 4.6) , (1.6 , 6.7).
Solution : The aim is to find the coefficients 𝑎3 , 𝑎2 , 𝑎1 , 𝑎0 of the equation 𝑦 = 𝑎3 𝑥 3 + 𝑎2 𝑥 2 + 𝑎1 𝑥 + 𝑎0 by solving the four equations:
𝟓 𝟓 𝟓 𝟓 𝟓

(∑ 𝒙𝒊 𝟎 ) 𝒂𝟎 + (∑ 𝒙𝒊 𝟏 ) 𝒂𝟏 + (∑ 𝒙𝒊 𝟐 ) 𝒂𝟐 + (∑ 𝒙𝒊 𝟑 ) 𝒂𝟑 = ∑ 𝒙𝒊 𝟎 𝒚𝒊
𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏
𝟓 𝟓 𝟓 𝟓 𝟓

(∑ 𝒙𝒊 𝟏 ) 𝒂𝟎 + (∑ 𝒙𝒊 𝟐 ) 𝒂𝟏 + (∑ 𝒙𝒊 𝟑 ) 𝒂𝟐 + (∑ 𝒙𝒊 𝟒 ) 𝒂𝟑 = ∑ 𝒙𝒊 𝟏 𝒚𝒊
𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏
𝟓 𝟓 𝟓 𝟓 𝟓

(∑ 𝒙𝒊 𝟐 ) 𝒂𝟎 + (∑ 𝒙𝒊 𝟑 ) 𝒂𝟏 + (∑ 𝒙𝒊 𝟒 ) 𝒂𝟐 + (∑ 𝒙𝒊 𝟓 ) 𝒂𝟑 = ∑ 𝒙𝒊 𝟐 𝒚𝒊
𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏
𝟓 𝟓 𝟓 𝟓 𝟓

(∑ 𝒙𝒊 𝟑 ) 𝒂𝟎 + (∑ 𝒙𝒊 𝟒 ) 𝒂𝟏 + (∑ 𝒙𝒊 𝟓 ) 𝒂𝟐 + (∑ 𝒙𝒊 𝟔 ) 𝒂𝟑 = ∑ 𝒙𝒊 𝟑 𝒚𝒊
𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏 𝒊=𝟏
To do so we prepare Table 4:
Table 4 : Example : Best fit for Non-linear data using Third Degree Polynomial

i 𝒙𝒊 𝒚𝒊 𝒙𝒊 𝟐 𝒙𝒊 𝟑 𝒙𝒊 𝟒 𝒙𝒊 𝟓 𝒙𝒊 𝟔 𝒙𝒊 𝒚𝒊 𝒙𝒊 𝟐 𝒚𝒊 𝒙𝒊 𝟑 𝒚𝒊
1 0.0 2.90 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
2 0.4 3.10 0.1600 0.0640 0.0256 0.0102 0.0041 1.2400 0.4960 0.1984
3 0.8 3.56 0.6400 0.5120 0.4096 0.3277 0.2621 2.8480 2.2784 1.8227
4 1.2 4.60 1.4400 1.7280 2.0736 2.4883 2.9860 5.5200 6.6240 7.9488
5 1.6 6.70 2.5600 4.0960 6.5536 10.4858 16.7772 10.7200 17.1520 27.4432
∑ → 4.0 20.86 4.8000 6.4000 9.0624 13.3120 20.0294 20.3280 26.5504 37.4131
Hence, the four equations to be solved are:
𝟓𝒂𝟎 + 𝟒𝒂𝟏 + 𝟒. 𝟖𝒂𝟐 + 𝟔. 𝟒𝒂𝟑 = 𝟐𝟎. 𝟖𝟔
𝟒𝒂𝟎 + 𝟒. 𝟖𝒂𝟏 + 𝟔. 𝟒𝒂𝟐 + 𝟗. 𝟎𝟔𝟐𝟒𝒂𝟑 = 𝟐𝟎. 𝟑𝟐𝟖
𝟒. 𝟖𝒂𝟎 + 𝟔. 𝟒𝒂𝟏 + 𝟗. 𝟎𝟔𝟐𝟒𝒂𝟐 + 𝟏𝟑. 𝟑𝟏𝟐𝒂𝟑 = 𝟐𝟔. 𝟓𝟓𝟎𝟒
𝟔. 𝟒𝒂𝟎 + 𝟗. 𝟎𝟔𝟐𝟒𝒂𝟏 + 𝟏𝟑. 𝟑𝟏𝟐𝒂𝟐 + 𝟐𝟎. 𝟎𝟐𝟗𝟒𝒂𝟑 = 𝟑𝟕. 𝟒𝟏𝟑𝟏
Solution of these equations yield 𝑎3 = 1.0417, 𝑎2 = −0.5446, 𝑎1 = 0.5798, 𝑎0 = 2.8977. Hence, the cubic function which best fits the
given set of data is 𝑦 = 1.0417𝑥 3 − 0.5446𝑥 2 + 0.5798𝑥 + 2.8977.

Page 11 of 15
 Figure 12: Example 4 : Comparing Best-fit curve for Non-linear data using Quadratic and Cubic Polynomial Functions

Figure 12 clearly shows that the cubic polynomial fit is superior to the quadratic one. The cubic polynomial fit almost goes through all five
data points, but not exactly. This is because a third-degree polynomial has four coefficients but the data contains five points. A fourth-degree
polynomial, which has five coefficients, would exactly agree with the data in this example.

In general, if a set of 𝑛 data points is given, then the (𝑛 − 1)𝑡ℎ − 𝑑𝑒𝑔𝑟𝑒𝑒 polynomial that best fits the data will agree exactly with the data
points. This is the main idea behind interpolation, and such polynomial is called an interpolating polynomial.

4.5. Polynomial Interpolation

Given a set of 𝑛 + 1 data points (𝑥1 , 𝑦1 ), … , (𝑥𝑛+1 , 𝑦𝑛+1 ), there is only one polynomial of degree at most 𝑛 in the form
𝒑(𝒙) = 𝒂𝒏+𝟏 𝒙𝒏 + 𝒂𝒏 𝒙𝒏−𝟏 + ⋯ + 𝒂𝟑 𝒙𝟐 + 𝒂𝟐 𝒙 + 𝒂𝟏
that goes through all the points. Although this polynomial is unique, it can be expressed in different forms. The two most commonly used
forms are provided by Lagrange Interpolating Polynomials and Newton Interpolating Polynomials.

Finite Differences Interpolation : Lagrange’s Interpolation

Lagrange Interpolating Polynomials are also known as Finite Differences Interpolation. The first-degree Lagrange Interpolating Polynomial
that goes through the two points (𝑥1 , 𝑦1 ) and (𝑥2 , 𝑦2 ) is in the form
𝒑𝟏 (𝒙) = 𝑳𝟏 (𝒙)𝒚𝟏 + 𝑳𝟐 (𝒙)𝒚𝟐
where 𝑳𝟏 (𝒙) and 𝑳𝟐 (𝒙) are the Lagrange coefficient functions and described by
𝒙 − 𝒙𝟐 𝒙 − 𝒙𝟏
𝑳𝟏 (𝒙) = , 𝑳𝟐 (𝒙) =
𝒙𝟏 − 𝒙𝟐 𝒙𝟐 − 𝒙𝟏
After substitution and solving, we have, 𝐿1 (𝑥1 ) = 1, 𝐿1 (𝑥2 ) = 0, 𝐿2 (𝑥1 ) = 0, and 𝐿2 (𝑥2 ) = 1. As a result, 𝑝1 (𝑥1 ) = 𝑦1 , and 𝑝1 (𝑥2 ) =
𝑦2 . This means that, the polynomial 𝑝1 (𝑥) is such that it passes through the two points (𝑥1 , 𝑦1 ) and (𝑥2 , 𝑦2 ) as a straight line. See Figure
13 (a).

Figure 13: Lagrange’s Interpolating Polynomials (a) First-degree Polynomial (b) Second-degree Polynomial

The second-degree Lagrange Interpolating Polynomial that goes through the three points (𝑥1 , 𝑦1 ) , (𝑥2 , 𝑦2 ) and (𝑥3 , 𝑦3 ) is in the form
𝒑𝟐 (𝒙) = 𝑳𝟏 (𝒙)𝒚𝟏 + 𝑳𝟐 (𝒙)𝒚𝟐 + 𝑳𝟑 (𝒙)𝒚𝟑

Page 12 of 15
Where
(𝒙 − 𝒙𝟐 )(𝒙 − 𝒙𝟑 ) (𝒙 − 𝒙𝟏 )(𝒙 − 𝒙𝟑 ) (𝒙 − 𝒙𝟏 )(𝒙 − 𝒙𝟐 )
𝑳𝟏 (𝒙) = , 𝑳𝟐 (𝒙) = 𝑳𝟑 (𝒙) =
(𝒙𝟏 − 𝒙𝟐 )(𝒙𝟏 − 𝒙𝟑 ) (𝒙𝟐 − 𝒙𝟏 )(𝒙𝟐 − 𝒙𝟑 ) (𝒙𝟑 − 𝒙𝟏 )(𝒙𝟑 − 𝒙𝟐 )
Then, 𝐿1 (𝑥1 ) = 1 = 𝐿2 (𝑥2 ) = 𝐿3 (𝑥3 ), while 𝐿𝑖 (𝑥𝑗 ) = 0 , 𝑓𝑜𝑟 𝑖 ≠ 𝑗. This guarantees 𝑝2 (𝑥1 ) = 𝑦1 , 𝑝2 (𝑥2 ) = 𝑦2 and 𝑝2 (𝑥3 ) = 𝑦3 , so
that the polynomial goes through the three points. See Figure 13 (b).

In general, the 𝑛𝑡ℎ − 𝑑𝑒𝑔𝑟𝑒𝑒 Lagrange Interpolating Polynomial that goes through 𝑛 + 1 data points (𝑥1 , 𝑦1 ), … , (𝑥𝑛+1 , 𝑦𝑛+1 ), is formed as
𝒏+𝟏

𝒑𝒏 (𝒙) = 𝑳𝟏 (𝒙)𝒚𝟏 + 𝑳𝟐 (𝒙)𝒚𝟐 + ⋯ + 𝑳𝒏+𝟏 (𝒙)𝒚𝒏+𝟏 = ∑ 𝑳𝒊 (𝒙)𝒚𝒊

𝒊=𝟏
where each 𝐿𝑖 (𝑥) is defined as
𝒏+𝟏
(𝒙 − 𝒙𝒋 )
𝑳𝒊 (𝒙) = ∏
𝒋=𝟏
(𝒙𝒊 − 𝒙𝒋 )
𝒋≠𝒊
and “Π” denotes the product of terms.

Example : Find the second-degree Lagrange interpolating polynomial for the data (0.1, 0.12) , (0.5, 0.47) , (0.9, 0.65). Use this polynomial
to find the interpolated value at 𝑥 = 0.3.

Solution : First, the three Lagrange coefficient functions are found as

(𝒙 − 𝒙𝟐 )(𝒙 − 𝒙𝟑 ) (𝒙 − 𝟎. 𝟓)(𝒙 − 𝟎. 𝟗) (𝒙 − 𝒙𝟏 )(𝒙 − 𝒙𝟑 ) (𝒙 − 𝟎. 𝟏)(𝒙 − 𝟎. 𝟗)
𝑳𝟏 (𝒙) = = , 𝑳𝟐 (𝒙) = = ,
(𝒙𝟏 − 𝒙𝟐 )(𝒙𝟏 − 𝒙𝟑 ) (𝟎. 𝟏 − 𝟎. 𝟓)(𝟎. 𝟏 − 𝟎. 𝟗) (𝒙𝟐 − 𝒙𝟏 )(𝒙𝟐 − 𝒙𝟑 ) (𝟎. 𝟓 − 𝟎. 𝟏)(𝟎. 𝟓 − 𝟎. 𝟗)
(𝒙 − 𝒙𝟏 )(𝒙 − 𝒙𝟐 ) (𝒙 − 𝟎. 𝟏)(𝒙 − 𝟎. 𝟓)
𝑳𝟑 (𝒙) = =
(𝒙𝟑 − 𝒙𝟏 )(𝒙𝟑 − 𝒙𝟐 ) (𝟎. 𝟗 − 𝟎. 𝟏)(𝟎. 𝟗 − 𝟎. 𝟓)
Then, the second-degree Lagrange polynomial is found as
𝒑𝟐 (𝒙) = 𝑳𝟏 (𝒙)𝒚𝟏 + 𝑳𝟐 (𝒙)𝒚𝟐 + 𝑳𝟑 (𝒙)𝒚𝟑
(𝒙 − 𝒙𝟐 )(𝒙 − 𝒙𝟑 ) (𝒙 − 𝒙𝟏 )(𝒙 − 𝒙𝟑 ) (𝒙 − 𝒙𝟏 )(𝒙 − 𝒙𝟐 )
= 𝒚 + 𝒚 + 𝒚
(𝒙𝟏 − 𝒙𝟐 )(𝒙𝟏 − 𝒙𝟑 ) 𝟏 (𝒙𝟐 − 𝒙𝟏 )(𝒙𝟐 − 𝒙𝟑 ) 𝟐 (𝒙𝟑 − 𝒙𝟏 )(𝒙𝟑 − 𝒙𝟐 ) 𝟑
(𝒙 − 𝟎. 𝟓)(𝒙 − 𝟎. 𝟗) (𝒙 − 𝟎. 𝟏)(𝒙 − 𝟎. 𝟗) (𝒙 − 𝟎. 𝟏)(𝒙 − 𝟎. 𝟓)
= ( 𝟎. 𝟏𝟐 ) + ( 𝟎. 𝟒𝟕 ) + ( 𝟎. 𝟔𝟓 )
(𝟎. 𝟏 − 𝟎. 𝟓)(𝟎. 𝟏 − 𝟎. 𝟗) (𝟎. 𝟓 − 𝟎. 𝟏)(𝟎. 𝟓 − 𝟎. 𝟗) (𝟎. 𝟗 − 𝟎. 𝟏)(𝟎. 𝟗 − 𝟎. 𝟓)
𝟐
= −𝟎. 𝟓𝟑𝟏𝟐𝒙 + 𝟏. 𝟏𝟗𝟑𝟕𝒙 + 𝟎. 𝟎𝟎𝟓𝟗
Using this polynomial, we can interpolate the value at 𝑥 = 0.3 as 𝑦 = −0.5312(0.3)2 + 1.1937(0.3) + 0.0059 = 0.3162.
Drawbacks of Lagrange Interpolation
With Lagrange interpolating polynomials, we learnt that going from one degree to the next, no information is stored from the lower-degree
polynomial and used in the construction of the new higher-degree polynomial. This is particularly inconvenient in two situations: (1) when the
exact degree of the interpolating polynomial is not known in advance; for instance, it might be better to use a portion of the set of data, or (2)
when additional points are added to the data. In these cases, a more suitable form is provided by the Newton interpolating polynomials.

Divided Differences Interpolation : Newton’s Interpolation

The Newton interpolating polynomials are recursively constructed as
𝒑𝟏 (𝒙) = 𝒂𝟏 + 𝒂𝟐 (𝒙 − 𝒙𝟏 )
𝒑𝟐 (𝒙) = 𝒂𝟏 + 𝒂𝟐 (𝒙 − 𝒙𝟏 ) + 𝒂𝟑 (𝒙 − 𝒙𝟏 )(𝒙 − 𝒙𝟐 ) = 𝒑𝟏 (𝒙) + 𝒂𝟑 (𝒙 − 𝒙𝟏 )(𝒙 − 𝒙𝟐 )
𝒑𝟑 (𝒙) = 𝒑𝟐 (𝒙) + 𝒂𝟒 (𝒙 − 𝒙𝟏 )(𝒙 − 𝒙𝟐 )(𝒙 − 𝒙𝟑 )
⋮
𝒑𝒏 (𝒙) = 𝒑𝒏−𝟏 (𝒙) + 𝒂𝒏+𝟏 (𝒙 − 𝒙𝟏 )(𝒙 − 𝒙𝟐 ) ⋯ (𝒙 − 𝒙𝒏 )
where the coefficients 𝑎1 , 𝑎1 , ⋯ , 𝑎𝑛 are determined inductively as follows:
Since 𝑝1 (𝑥) must agree with the data at (𝑥1 , 𝑦1 ) and (𝑥2 , 𝑦2 ) , we have
𝒑𝟏 (𝒙𝟏 ) = 𝒚𝟏 , and 𝒑𝟏 (𝒙𝟐 ) = 𝒚𝟐
so that
𝒂𝟏 + 𝒂𝟐 (𝒙𝟏 − 𝒙𝟏 ) = 𝒚𝟏 𝒚𝟐 − 𝒚𝟏
⇒ 𝒂 𝟏 = 𝒚𝟏 , 𝒂 𝟐 =
𝒂𝟏 + 𝒂𝟐 (𝒙𝟐 − 𝒙𝟏 ) = 𝒚𝟐 𝒙𝟐 − 𝒙𝟏
Similarly, 𝑝2 (𝑥) must agree with the data at (𝑥1 , 𝑦1 ) , (𝑥2 , 𝑦2 ) and (𝑥3 , 𝑦3 ) ; hence
𝒑𝟐 (𝒙𝟏 ) = 𝒚𝟏 , 𝒑𝟐 (𝒙𝟐 ) = 𝒚𝟐 and 𝒑𝟐 (𝒙𝟑 ) = 𝒚𝟑
This gives
𝒚𝟑 − 𝒚𝟐 𝒚 𝟐 − 𝒚𝟏
𝒚 𝟐 − 𝒚𝟏 −
𝒙 − 𝒙𝟐 𝒙𝟐 − 𝒙𝟏
𝒂 𝟏 = 𝒚𝟏 , 𝒂 𝟐 = , 𝒂𝟑 = 𝟑
𝒙𝟐 − 𝒙𝟏 𝒙𝟑 − 𝒙𝟏
Continuing this process yields all remaining coefficients. These coefficients follow a systematic pattern that is best understood through
Newton’s divided differences. For two points (𝑥1 , 𝑦1 ) and (𝑥2 , 𝑦2 ) , the first divided difference is the slope of the line connecting them and
denoted by
𝒚𝟐 − 𝒚𝟏
𝒇[ 𝒙𝟐 , 𝒙𝟏 ] = = 𝒂𝟐
𝒙𝟐 − 𝒙𝟏
Page 13 of 15
For three points (𝑥1 , 𝑦1 ) , (𝑥2 , 𝑦2 ) and (𝑥3 , 𝑦3 ) , the second divided difference is defined as
𝒚𝟑 − 𝒚𝟐 𝒚 𝟐 − 𝒚𝟏
𝒇[ 𝒙𝟑 , 𝒙𝟐 ] − 𝒇[ 𝒙𝟐 , 𝒙𝟏 ] −
𝒙 − 𝒙𝟐 𝒙𝟐 − 𝒙𝟏
𝒇[ 𝒙𝟑 , 𝒙𝟐 , 𝒙𝟏 ] = = 𝟑 = 𝒂𝟑
𝒙𝟑 − 𝒙𝟏 𝒙𝟑 − 𝒙𝟏
In general, the 𝑘 𝑡ℎ divided difference is described by
𝒇[ 𝒙𝒌+𝟏 , 𝒙𝒌 , ⋯ , 𝒙𝟒 , 𝒙𝟑 , 𝒙𝟐 ] − 𝒇[ 𝒙𝒌 , 𝒙𝒌−𝟏 , ⋯ , 𝒙𝟑 , 𝒙𝟐 , 𝒙𝟏 ]
𝒇[ 𝒙𝒌+𝟏 , 𝒙𝒌 , ⋯ , 𝒙𝟑 , 𝒙𝟐 , 𝒙𝟏 ] = = 𝒂𝒌+𝟏
𝒙𝒌+𝟏 − 𝒙𝟏
Therefore, the 𝑛𝑡ℎ − 𝑑𝑒𝑔𝑟𝑒𝑒 Newton divided-difference interpolating polynomial for the data (𝑥1 , 𝑦1 ), (𝑥2 , 𝑦2 ), ⋯ , (𝑥𝑛+1 , 𝑦𝑛+1 ) is formed as
𝒑𝒏 (𝒙) = 𝒂𝟏 + 𝒂𝟐 (𝒙 − 𝒙𝟏 ) + 𝒂𝟑 (𝒙 − 𝒙𝟏 )(𝒙 − 𝒙𝟐 ) + ⋯ + 𝒂𝒏+𝟏 (𝒙 − 𝒙𝟏 )(𝒙 − 𝒙𝟐 ) ⋯ (𝒙 − 𝒙𝒏 )
where
𝒂 𝟏 = 𝒚𝟏 , 𝒂𝟐 = 𝒇[ 𝒙𝟐 , 𝒙𝟏 ], 𝒂𝟑 = 𝒇[ 𝒙𝟑 , 𝒙𝟐 , 𝒙𝟏 ], ⋯, 𝒂𝒏+𝟏 = 𝒇[ 𝒙𝒏+𝟏 , 𝒙𝒏 , ⋯ , 𝒙𝟑 , 𝒙𝟐 , 𝒙𝟏 ]
The coefficients 𝑎1 , 𝑎1 , ⋯ , 𝑎𝑛 are best calculated with the aid of a Divided Differences Table. A typical such table is presented in Table 5.7
corresponding to a set of five data points.
Table 5 : Divided Differences Table for five data points
First Divided Second Divided Third Divided Fourth Divided
𝒊 𝒙𝒊 𝒚𝒊
Difference Difference Difference Difference
1 𝑥1 𝑦1 = 𝑎1
𝑓[𝑥2 , 𝑥1 ] = 𝑎2
2 𝑥2 𝑦2 𝑓[𝑥3 , 𝑥2 , 𝑥1 ] = 𝑎3
𝑓[𝑥3 , 𝑥2 ] 𝑓[𝑥4 , 𝑥3 , 𝑥2 , 𝑥1 ] = 𝑎4
3 𝑥3 𝑦3 𝑓[𝑥4 , 𝑥3 , 𝑥2 ] 𝑓[𝑥5 , 𝑥4 , 𝑥3 , 𝑥2 , 𝑥1 ] = 𝑎5
𝑓[𝑥4 , 𝑥3 ] 𝑓[𝑥5 , 𝑥4 , 𝑥3 , 𝑥2 ]
4 𝑥4 𝑦4 𝑓[𝑥5 , 𝑥4 , 𝑥3 ]
𝑓[𝑥4 , 𝑥3 ]
5 𝑥5 𝑦5
Example : Find the fourth-degree Newton interpolating polynomial for the data (0, 0) , (0.1, 0.1210) , (0.2, 0.2258) ,
(0.5, 0.4650), (0.8, 0.6249). Use this polynomial to find the interpolated value at 𝑥 = 0.7.
Solution : For the given data divided differences are calculated in Table 6. From the table we can find the Newton’s fourth-degree
interpolating polynomial as:
𝑝4 (𝑥) = 𝑎1 + 𝑎2 (𝑥 − 𝑥1 ) + 𝑎3 (𝑥 − 𝑥1 )(𝑥 − 𝑥2 ) + 𝑎4 (𝑥 − 𝑥1 )(𝑥 − 𝑥2 )(𝑥 − 𝑥3 ) + 𝑎5 (𝑥 − 𝑥1 )(𝑥 − 𝑥2 )(𝑥 − 𝑥3 )(𝑥 − 𝑥4 )
= 0 + 1.21(𝑥 − 0) − 0.81(𝑥 − 0)(𝑥 − 0.1) + 0.3664(𝑥 − 0)(𝑥 − 0.1)(𝑥 − 0.2) − 0.1254(𝑥 − 0)(𝑥 − 0.1)(𝑥 − 0.2)(𝑥 − 0.5)
= −0.1254𝑥 4 + 0.4667𝑥 3 − 0.9412𝑥 2 + 1.2996𝑥
Using this polynomial the interpolated value at 𝑥 = 0.7 is 𝑦 = −0.1254(0.7)4 + 0.4667(0.7)3 − 0.9412(0.7)2 + 1.2996(0.7) = 0.5784
Table 6 : Divided Differences Table for example problem
First Divided Second Divided Third Divided Fourth Divided
𝒊 𝒙𝒊 𝒚𝒊
Difference Difference Difference Difference
1 𝑥1 = 0.0 (𝑦1 = 0.0) = 𝑎1
𝑓[𝑥2 , 𝑥1 ] = 𝑎2
0.1210 − 0
= = 1.21
0.1 − 0
𝑓[𝑥3 , 𝑥2 , 𝑥1 ] = 𝑎3
2 𝑥2 = 0.1 𝑦2 = 0.1210 1.0480 − 0.2100
= = −0.8100
0.2 − 0.0
𝑓[𝑥4 , 𝑥3 , 𝑥2 , 𝑥1 ] = 𝑎4
𝑓[𝑥3 , 𝑥2 ]
−0.6268 + 0.8100
0.2258 − 0.1210 =
= = 1.0480 0.5 − 0.0
0.2 − 0.1 = 0.3664
𝑓[𝑥5 , 𝑥4 , 𝑥3 , 𝑥2 , 𝑥1 ] = 𝑎5
𝑓[𝑥4 , 𝑥3 , 𝑥2 ]
0.2661 − 0.3664
3 𝑥3 = 0.2 𝑦3 = 0.2258 0.7973 − 1.0480 =
= = −0.6268 0.8 − 0.0
0.5 − 0.2 = −0.1254
𝑓[𝑥5 , 𝑥4 , 𝑥3 , 𝑥2 ]
𝑓[𝑥4 , 𝑥3 ]
−0.4405 + 0.6268
0.4650 − 0.2258 =
= = 0.7973 0.8 − 0.1
0.5 − 0.2 = 0.2661
𝑓[𝑥5 , 𝑥4 , 𝑥3 ]
4 𝑥4 = 0.5 𝑦4 = 0.4650 0.5330 − 0.7973
= = −0.4405
0.8 − 0.5
𝑓[𝑥5 , 𝑥4 ]
0.6249 − 0.4650
= = 0.5330
0.8 − 0.5
5 𝑥5 = 0.8 𝑦5 = 0.6249

Page 14 of 15
Page 15 of 15