Infrared Analysis AACC International Method 39-01.

01
Page 1 of 6

Evaluation of NIR Instrument Calibration

Final approval November 17, 2010

Objective
This procedure (see Note 1) covers the general protocol to be adopted when
an instrument’s calibration is statistically assessed. It assumes that reference and
instrument values have been collected from a common set of samples by an
appropriate method. By using regression analysis, the coefficients of the linear
relationship between the instrument values and reference values of a set of
standard samples are determined. When the reference values are regressed onto
the instrument values, the slope is tested to determine whether it is significantly
different from a 45° line (for skew), and if it is not significantly different, the
significance of the bias (instrument mean minus reference mean) is tested.
Where appropriate, precision and accuracy are also assessed according to the
procedures specified in Method 39-00.01.

Apparatus
1. Reference and measured values on at least 11 samples (with the tables
contained herein listing cutoff values for 9–30 samples, after removal of out-
liers).
2. Calculator or, preferably, a personal computer loaded with an application
program for spreadsheet analysis capable of performing simple linear regres-
sion, such as Microsoft Excel.

Procedure
The standard deviation of reference values in the evaluation set should be at
least four times the original standard error of prediction of the calibration. The
reference values for the evaluation set should come from the same laboratory
that generated the original calibration data or be statistically equivalent, as
demonstrated by a valid check or collaborative sample program. The following
steps are applicable to spreadsheet and statistical software up to the stage of
calculating the equation of the line of best fit. Checking for outliers, significance
testing, and precision and accuracy calculations are readily carried out with
user-entered formulae (given below) in a spreadsheet. A worked example is
given below (see Calculations section). Analysis conducted with statistical
software is also acceptable.

Linear Regression
Construct a column of data containing the reference values (to be assigned to
the dependent variable – y input range) and a second (adjacent) column con-
taining the corresponding instrument values (to be assigned to the independent
variable – x input range) in the same row.

http://dx.doi.org/10.1094/AACCIntMethod-39-01.01
Infrared Analysis AACC International Method 39-01.01
Page 2 of 6

Evaluation of NIR Instrument Calibration (continued)

Testing for Outliers

Outliers are defined here as experimental values whose residuals with respect
to the reference values lie more than 3 standard deviations from their regression
line. See Note 2.
Carry out linear regression analysis to provide the coefficients (alternatively
named constants) a and b in the equation
y = a+b× x
where y is the reference value and x is the instrument value. The constant a will
be in a row denoted by a term such as “intercept,” and the coefficient b will be
in a row denoted by a term such as “X variable 1” or “slope”. The formulae for
calculating a and b are

b=
∑ (x − x )(y − y ) = n∑ x y − ∑ x ∑ y
i i i i i i

∑ (x − x ) n∑ x − (∑ x )
2 2 2
i i i

a = y − bx
where xi = instrument predicted value for sample i; yi = reference value for
sample i; x = mean of the instrument predicted values (x bar); y = mean of the
reference values (y bar); n = number of samples; and Σ = summation from l to n.
Of the two equivalent forms given for b, the first is the more suitable for
programming on a computer because it is less prone to rounding errors, while
the second is easier for manual calculation, provided care is taken to carry
sufficient significant figures in intermediate stages of the calculation.
Alternatively, the direct functional expressions in Microsoft Excel are
INTERCEPT(y1:yn, x1:xn) for a and SLOPE(y1:yn, x1:xn) for b, where y1:yn and
x1:xn refer to the spreadsheet cell ranges.
For each value, calculate the estimate of the reference value, ŷ, from the
equation:
yˆ = a + b × x
Calculate the difference between the reference and estimate of reference
values (y – ŷ ). These values are known as the residuals. For each sample,
calculate the standardized residual (see below), which is defined here as
y − yˆ
s
where

s=
∑ (y − yˆ )
i i
2

=
∑ ( y − y ) − b ∑ (x − x )
i
2 2
i
2

n−2 n−2
and yi = reference value for sample i; y = mean of the reference values (y bar);
ŷi = estimate of the reference value for sample i; xi = instrument value for
Infrared Analysis AACC International Method 39-01.01
Page 3 of 6

Evaluation of NIR Instrument Calibration (continued)

sample i; x = mean of the instrument values (x bar); n = number of points; and

Σ = summation from 1 to n.
The value s may be given directly by the software package denoted by a term
similar to “standard error of the estimate.” For example, in Excel, the function
STEYX can be used with the original values for x and y as input, e.g.,
STEYX(y1:yn, x1:xn). See Note 3.
Divide each residual value by s, giving the standardized residual. Any
samples with absolute standardized residual values ≥3 are considered outliers
and should be removed from the data set.

Line of Best Fit

If there are no outliers, then the constants required are a and b as calculated
above. If there are outliers, then, with the remaining data, carry out a linear
regression analysis to produce revised values for a and b. (Note that with
respect to the original value of n, the new value of n is reduced by the number
of outliers removed).

Testing the Slope, b, for Skew

Test the significance of the slope by the Student’s t test using the formula
1− b
t=
s ∑ (x − x )
i
2

Critical values of t when testing the skew are given in Table 1.

If the absolute value of t is greater than or equal to the corresponding critical
value in Table 1, then the calibration is skewed and the question of bias does not
arise. In this event and in order of preference, the instrument should be recali-
brated or the predictions from the calibration (xi) can be multiplied by the slope,
b, and the intercept, a, is added to this product.

Table 1. Critical Values of t, at the 5% Level,a When Testing the Skew

No. of samples after
removal of outliers 9 10 11 12 13 14 15 16 17 18 19
Degrees of freedom 7 8 9 10 11 12 13 14 15 16 17
Critical t value 2.37 2.31 2.26 2.23 2.20 2.18 2.16 2.15 2.13 2.12 2.11

No. of samples after

removal of outliers 20 21 22 23 24 25 26 27 28 29 30
Degrees of freedom 18 19 20 21 22 23 24 25 26 27 28
Critical t value 2.10 2.09 2.09 2.08 2.07 2.07 2.06 2.06 2.06 2.05 2.05
a Data from Snedecor and Cochran (Ref. 4).
Infrared Analysis AACC International Method 39-01.01
Page 4 of 6

Evaluation of NIR Instrument Calibration (continued)

Testing for Bias

If a slope adjustment is not required, then test the significance of the bias
using the Student’s t test.

t=
(∑ d )/ n i

∑ d − (∑ d )
2
i
2
i n
n(n − 1)
where di = xi – yi, measured instrument value minus reference value for the ith
sample.
The bias is (Σdi)/n. Critical values of t when testing the bias are given in Table 2.
If the absolute value of t is greater than or equal to the corresponding critical
value in Table 2, then the calibration is biased. The calibration can be corrected
by subtracting the bias from the instrument value, xi.

Calculation of Accuracy
Accuracy, the root mean squared deviation (RMSD), is calculated using the
formula

RMSD =
∑d i
2

n
In the worked example, the RMSD appears in the bottom entry of the column of
squared differences.

Calculation of Precision
Precision is the standard deviation of replicates (SDR) given by the formula

SDR =
∑w 2
i

2n
where wi = the difference between pairs of instrument readings for sample i and
n = the number of pairs of reading.

Table 2. Critical Values of t, at the 5% Level,a When Testing the Bias

No. of samples after
removal of outliers 9 10 11 12 13 14 15 16 17 18 19
Degrees of freedom 8 9 10 11 12 13 14 15 16 17 18
Critical t value 2.31 2.26 2.23 2.20 2.18 2.16 2.15 2.13 2.12 2.11 2.10

No. of samples after

removal of outliers 20 21 22 23 24 25 26 27 28 29 30
Degrees of freedom 19 20 21 22 23 24 25 26 27 28 29
Critical t value 2.09 2.09 2.08 2.07 2.07 2.06 2.06 2.06 2.05 2.05 2.04
a Data from Snedecor and Cochran (Ref. 4).
Infrared Analysis AACC International Method 39-01.01
Page 5 of 6

Evaluation of NIR Instrument Calibration (continued)

Calculations
The following spreadsheet is a worked example that utilizes 20 samples and is
described under Procedure.

A B C D E F G H I J K
1 User Calculations
2 2 2
y x y hat y - y hat (y - y hat) (y - y hat)/s x - xbar (x - xbar) di di
2
Standard- Instrument
Estimated Residual ised minus Difference
3 Sample Number Reference Instrument Value of y Residual Squared Residual Reference Squared
4 1 11.83 11.60 11.89 -0.06 0.0036 -0.35 -3.2165 10.3459 -0.23 0.0529
5 2 12.08 11.71 11.99 0.09 0.0079 0.52 -3.1065 9.6503 -0.37 0.1369
6 3 12.28 11.85 12.12 0.16 0.0257 0.93 -2.9665 8.8001 -0.43 0.1849
7 4 12.62 12.52 12.74 -0.12 0.0135 -0.68 -2.2965 5.2739 -0.10 0.0100
8 5 13.08 13.03 13.21 -0.13 0.0156 -0.73 -1.7865 3.1916 -0.05 0.0025
9 6 13.33 13.21 13.37 -0.04 0.0016 -0.24 -1.6065 2.5808 -0.12 0.0144
10 7 13.72 13.49 13.63 0.09 0.0085 0.54 -1.3265 1.7596 -0.23 0.0529
11 8 13.88 13.83 13.94 -0.06 0.0037 -0.35 -0.9865 0.9732 -0.05 0.0025
12 9 14.24 14.07 14.16 0.08 0.0062 0.46 -0.7465 0.5573 -0.17 0.0289
13 10 14.75 14.90 14.92 -0.17 0.0306 -1.02 0.0835 0.0070 0.15 0.0225
14 11 15.00 14.90 14.92 0.08 0.0057 0.44 0.0835 0.0070 -0.10 0.0100
15 12 15.24 15.44 15.42 -0.18 0.0329 -1.06 0.6235 0.3888 0.20 0.0400
16 13 15.65 15.83 15.78 -0.13 0.0169 -0.76 1.0135 1.0272 0.18 0.0324
17 14 16.04 15.82 15.77 0.27 0.0724 1.57 1.0035 1.0070 -0.22 0.0484
18 15 16.40 16.75 16.63 -0.23 0.0511 -1.32 1.9335 3.7384 0.35 0.1225
19 16 16.75 16.60 16.49 0.26 0.0685 1.53 1.7835 3.1809 -0.15 0.0225
20 17 17.09 17.45 17.27 -0.18 0.0324 -1.05 2.6335 6.9353 0.36 0.1296
21 18 17.40 17.25 17.09 0.31 0.0986 1.83 2.4335 5.9219 -0.15 0.0225
22 19 17.68 17.78 17.57 0.11 0.0114 0.62 2.9635 8.7823 0.10 0.0100
23 20 17.90 18.30 18.05 -0.15 0.0230 -0.88 3.4835 12.1348 0.40 0.1600
24 x bar 14.82 sum of squares, SS 0.5297 sum 86.26 -0.63 1.11
25 mean square, MS 0.0294 square root 9.2878
26 number 20 square root, s 0.1715 square 0.3969
27 bias -0.03
28 RMSD0.235
29 t skew 4.35 Action possibly required if absolute value over 2.0 (see Table 1)
30 t bias -0.59 If no skew adjustment then action possibly required if absolute value over 2.0 (see Table 2)
31
32 Abbreviated output from the regression analysis
33 SUMMARY OUTPUT
34
35 Regression Statistics Note: where user and regression
36 Multiple R 0.9964 calculations correspond
37 R Square 0.9928 cell F24 = cell C45
38 Adjusted R Square 0.9924 cell F25 = cell D45
39 Standard Error 0.1715 cell F26 = cell B39
40 Observations 20
41
42 ANOVA
43 df SS MS
44 Regression 1 72.9593 72.9593
45 Residual 18 0.5297 0.0294
46 Total 19 73.4889
47
Coeff-
48 icients Constant
49 Intercept 1.2219 a
50 X Variable 1 0.9197 b
Infrared Analysis AACC International Method 39-01.01
Page 6 of 6

Evaluation of NIR Instrument Calibration (continued)

Notes
1. This method, with modifications, is reprinted with permission from the
Campden BRI (formerly Campden & Chorleywood Food Research Association
[CCFRA]) Flour Testing Working Group’s Guideline No. 3 (Ref. 1) and as
described by Osborne and Fearn (Ref. 3).
2. The standardized residual should be calculated as detailed herein. If a
spreadsheet software package calculates values designated as such, cross check
the calculation before using the values. Dedicated statistics packages that per-
form more-involved calculations to check for outliers may also be used.
3. In the example, cell B26 contains the number of samples, n, and is refer-
enced whenever n is required in a calculation. If there is an analysis of variance
(ANOVA) table, then SS = the value in the cell defined by the residual row and
the sum of squares (SS) column and MS = the value in the cell defined by the
residual row and the mean square (MS) column. Note that s is the square root of
this value.

References
1. Campden & Chorleywood Food Research Association. 2002. Assessing an Instrument Calibration.
Flour Testing Working Group Procedure No. 02p. Manual of Methods for Wheat and Flour
Testing, Guideline No. 3. The Association, Gloucestershire, UK.
2. Delwiche, S. R., and Millar, S. 2010. A new AACC International approved method for assessment
of a calibration from a near infrared spectrometer. Cereal Foods World 55:293-296.
3. Osborne, B. G., and Fearn, T. 1986. Near Infrared Spectroscopy in Food Analysis, pp. 186-190.
Longman Scientific and Technical, Essex, UK.
4. Snedecor, G. W., and Cochran, W. G. 1989. Statistical Methods, 8th ed. Iowa State University
Press, Ames.

Calculator: Near Infrared Calibration Evaluation Spreadsheet