Conducting An Interlaboratory Study To Determine The Precision of A Test Method
Conducting An Interlaboratory Study To Determine The Precision of A Test Method
Conducting An Interlaboratory Study To Determine The Precision of A Test Method
for the
Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
This standard has been approved for use by agencies of the U.S. Department of Defense.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
4.1.2 Precision is reported as a standard deviation, coeffi- Precision Statement Information Section
cient of variation (relative standard deviation), variance, or a Repeatability and Reproducibility 21
precision limit (a data range indicating no statistically signifi-
Section
cant difference between test results). Keywords 22
4.1.3 This practice is designed only to estimate the precision
of a test method. However, when accepted reference values are Tables Table
Glucose in Serum Example 1–4, 6–8
available for the property levels, the test result data obtained Critical Values of Consistency Statistics, h and k 5
according to this practice may be used in estimating the bias of
the test method. For a discussion of bias estimation and the Figures Figure
Glucose in Serum Example 1–3
relationships between precision, bias, and accuracy, see Prac-
tice E177. Annexes Annex
Theoretical Considerations Annex A1
4.2 The procedures presented in this practice consist of Calculation of the ILS Statistics for Unbalanced Data Sets Annex A2
three basic steps: planning the interlaboratory study, guiding
Appendixes Appendix
the testing phase of the study, and analyzing the test result data. Spreadsheet for E691 Calculations Appendix X1
4.2.1 The planning phase includes forming the ILS task
group, the study design, selection, and number of participating 5. Concepts of Test Method Precision
laboratories, selection of test materials, material certifications
if applicable, and writing the ILS protocol. A well-developed 5.1 Repeatability and Reproducibility—These two terms
test method is essential, so including a ruggedness test to deal with the variability of test results obtained under specified
determine control of test method conditions is highly recom- laboratory conditions and represent the two extremes of test
mended. method precision. Repeatability concerns the variability be-
tween independent test results obtained within a single labo-
NOTE 1—In this practice, the term test method is used both for the actual
ratory in the shortest practical period of time by a single
measurement process and for the written description of the process, while
the term protocol is used for the directions given to the laboratories for operator with a specific set of test apparatus using test
conducting the ILS. specimens (or test units) taken at random from a single quantity
4.2.2 The testing phase includes material preparation and of homogeneous material obtained or prepared for the ILS.
distribution, liaison with the participating laboratories, and Reproducibility deals with the variability between single test
handling of test result data received from the laboratories. results obtained in different laboratories, each of which has
4.2.3 The data analysis utilizes tabular, graphical, and sta- applied the test method to test specimens (or test units) taken
tistical diagnostic tools for evaluating the consistency of the at random from a single quantity of homogeneous material
data so that unusual values may be detected and investigated, obtained or prepared for the ILS.
and also includes the calculation of the numerical measures of 5.1.1 Repeatability Conditions—The single-operator,
precision of the test method pertaining to repeatability and single-set-of-apparatus requirement means that for a particular
reproducibility. step in the measurement process the same combination of
4.3 The information in this practice is arranged as follows: operator and apparatus is used for every test result and on every
material. Thus, one operator may prepare the test specimens, a
Section
Scope 1 second measure the dimensions and a third measure the
Referenced Documents 2 breaking force. “Shortest practical period of time” means that
Terminology 3 the test results, at least for one material, are obtained in a time
Significance and Use 4
Concepts of Test Method Precision 5 not less than in normal testing and not so long as to permit
significant changes in test material, equipment or environment.
Planning the Interlaboratory Study (ILS) Section
ILS Membership 6 5.1.2 Reproducibility Conditions—The factors that contrib-
Basic Design 7 ute to variability in a single laboratory, such as operator,
Test Method 8 equipment used, calibration of the equipment, and environment
TABLE 2 Interlaboratory Study Worksheet for Glucose in Serum Initial Preparation of Test Result Data for Material C
Laboratory Test Results, x
x̄ s d h k
Number 1 2 3
1 132.66 133.83 133.10 133.197 0.591 –1.946 –0.73 0.22
2 132.92 136.90 136.40 135.407 2.168 0.264 0.10 0.79
3 132.61 135.80 135.36 134.590 1.729 –0.553 –0.21 0.63
4 138.50 148.30 135.69 140.830 6.620 5.687 2.14 2.41
5 131.90 134.14 133.76 133.267 1.199 –1.876 –0.71 0.44
6 137.21 135.14 137.50 136.617 1.287 1.474 0.55 0.47
7 130.97 131.59 134.92 132.493 2.124 –2.650 –1.00 0.77
8 135.46 135.14 133.63 134.743 0.977 –0.400 –0.15 0.36
where:
x = individual test result (see 15.3),
x̄ = cell average (see 15.4.1),
s = cell standard deviation (see 15.4.2),
% =
x average of cell averages (see 15.5.1),
d = cell deviation (see 15.5.2),
s x̄ = standard deviation of cell averages (see 15.5.3),
sr = repeatability standard deviation (see 15.6.1),
sL = between-laboratory standard deviation (see 15.6.2),
sR = reproducibility standard deviation (see 15.6.3),
h = between-laboratory consistency (see 15.7.1), and
k = within-laboratory consistency (see 15.7.2).
Œ(
~ 132.92 1 136.90 1 136.40!
x̄ 5 5 135.407 p
3
sr 5 s 2 /p (6)
1
15.4.2 Cell Standard Deviation, s—Calculate the standard
deviation of the test results in each cell using the following where:
equation: sr = the repeatability standard deviation,
Œ(
s = the cell standard deviation (p of them from Eq 2), and
n
p = the number of laboratories.
s5 ~ x 2 x̄ ! 2 / ~ n 2 1 ! (2)
1
Thus, for Material C:
The symbols have the same meaning as for Eq 1. Thus, for
Cell C2: sr 5 Œ 60.425223
8
5 =7.553153 5 2.7483
Œ(
2 −0.13 −0.45 0.10 0.15 1.64
p 3 −0.11 0.22 −0.21 −1.01 −0.68
s x̄ 5 d 2/ ~ p 2 1 ! (5) 4 −0.10 1.85 2.14 0.96 0.49
1 5 −0.09 −0.99 −0.71 −0.64 −0.34
6 0.83 0.21 0.55 0.97 0.17
Thus, for Material C: 7 −1.75 −0.16 −1.00 −1.33 −1.62
Œ
8 1.75 0.67 −0.15 1.31 0.79
49.376634
5 =7.053805 5 2.6559
A
Critical value = 2.15.
s x̄ 5
~8 2 1!
18. Investigation
TABLE 7 Glucose in Serum-kA,B
Material 18.1 Clerical and Sampling Errors—Examine the labora-
Laboratory tory report for each flagged cell. Try to locate where each test
A B C D E
1 0.21 0.11 0.38 0.02 0.18 result in the flagged cell begins to deviate from the others. Is it
2 0.46 0.89 1.40 1.78 2.33 in the original observations? Are the data rounded prema-
3 1.00 0.56 1.12 0.61 0.69 turely? Are the calculations correct? Then, look for signs of
4 1.70 1.85 1.02 0.74 0.22
5 0.34 0.52 0.78 0.72 0.24 mislabeling of test units such that the test result for one
6 1.32 1.09 0.83 0.63 1.03 material was reported as belonging to another material. Check
7 1.17 1.38 1.38 1.45 0.84 these errors with the laboratories: do not assume them to be so.
8 0.77 0.34 0.63 0.94 0.42
A
Recalculated values after correcting Cell C4 (see 20.1.4 and 20.1.5). 18.2 Procedural Errors:
B
Critical value = 2.06. 18.2.1 Study the laboratory reports again looking for devia-
tions from either the test method or the protocol. For instance,
variations in the number of significant digits reported in the test
for all or most of the materials. High k values represent results may be a sign of incorrect rounding, or that the
within-laboratory imprecision. Very small k values may indi- equipment in one laboratory is different from the rest. Also,
cate a very insensitive measurement scale or other measure- study the event log for special comments relating to the flagged
ment problem. cells.
19. Task Group Actions percent of the ILS data likely will lead to the presentation of
19.1 General—If the investigation disclosed no clerical, precision data that the test method cannot deliver in routine
sampling or procedural errors, the unusual data should be application.
retained, and the precision statistics based on them should be 19.3 Test Method Vagueness—One of the important things
published. If, on the other hand, a cause was found during the to be on the alert for during a laboratory investigation is for
investigation, the task group has several options to consider. If vagueness in the test method standard that permits a wide range
the laboratory clearly and seriously deviated from the test of interpretation leading to loss of precision. Particular ele-
method, the test results for that laboratory must be removed ments to check are lack of measurement tolerances, diversity of
from the ILS calculations. However, despite the danger of the apparatus and insufficient direction for operator technique.
recalcitrant laboratory having prior knowledge, it may be These problems can be the basis for a revision of the standard.
appropriate to ask the laboratory to retest one or more materials
following the correct procedure, and then include the new set 20. Glucose ILS Consistency
of test results in the ILS calculations. Of course, if the data 20.1 Glucose in Serum—The ILS is described in 15.1.1.
have changed, recalculation of the h and k values must be made 20.1.1 h Statistic—The overall impression given by Fig. 1
and the data consistency examined again. and Table 3 is one of reasonable consistency for variation
19.2 Exception—When a large number of laboratories have among laboratories. Only Laboratory 4 stands out with large
participated in the ILS and no causes for some unusual cell values for Materials B and C.
values have been found during the investigation, it may be 20.1.2 k Statistic—Laboratories 2 and 4 stand out in Fig. 2
appropriate to delete a cell from the study if all of the other and Table 4.
laboratories are in substantial agreement. The number of 20.1.3 Cells and Test Results—Cells C4 and E2 should be
laboratories that can be considered large enough to support investigated. A look at Table 1 reveals that the second test
deletion of data without an identified cause cannot be stated results of 148.30 in C4 and of 309.40 in E2 are the particular
exactly. Any action which results in discarding more than ten values to be investigated.
20.1.4 Action—If the data from Laboratory 4 were typed, 21.2 Prepare a table for the corrected precision statistics as
the result 148.30 in Cell C4 could have been a typographical shown in Table 8.
error. We have no way of knowing this today, many years after
21.3 Variation of Precision Statistics with Property Level:
this study was made. We will suppose, however, that the task
group did indeed call the laboratory and did find that the 21.3.1 Quite often the values of sr and sR will be found to
number should have been 138.30. However, let us suppose that vary with the values of the property level x% . This type of
for Cell E2 the task group could find no explanation of the response can be seen in Fig. 3, that is based on Table 8. The
apparently high value of 309.40. In such a case they should manner in which the statistics vary with the property level
retain the value. should be shown in presenting the precision information in the
20.1.5 Recalculation—Table 6 and Table 7 show the recal- precision statement of the test method. The statistician should
culated consistency statistics resulting from correcting Cell C4. recommend the most appropriate relationship to present, using
Practice E177 as a guide.
PRECISION STATEMENT INFORMATION 21.4 Precision Statement—Table 8 (with the column for s x̄
omitted) is a useful format for the presentation of the precision
21. Repeatability and Reproducibility statement of the test method as required by Section A21 of the
21.1 General—Once the task group has concluded which Form and Style of ASTM Standards2 (Bluebook). Having
cells are sufficiently inconsistent to require action, and action obtained the required precision information in accordance with
has been taken, the statistics of 15.4 through 15.6 are recalcu- this practice, the final form of the precision statement may be
lated (see also 20.1.5). Using the corrected statistics, calculate prepared in accordance with Practice E177.
for each material the 95 % repeatability and reproducibility
limits (see Practice E177) according to the following Eq 12 and 21.5 Conclusion—The precision statistics obtained by an
Eq 13: ILS such as described in this practice must not be treated as
exact mathematical quantities which are applicable to all
r 5 2.8 s r (12) circumstances and uses. The small number of laboratories and
R 5 2.8 s R (13) of materials included in the usual ILS guarantees that there will
FIG. 3 Glucose in Serum: Standard Deviations of Reproducibility (o) and Repeatability (•) Versus Average
be times when differences greater than predicted by the ILS 22. Keywords
results will arise, sometimes with considerably greater or 22.1 precision; repeatability; reproducibility; repeatability
smaller frequency than the 95 % probability limit would imply. limit; reproducibility limit
The repeatability limit and the reproducibility limit should be
considered as general guides, and the associated probability of
95 % as only a rough indicator of what can be expected. If
more precise information is needed in specific circumstances,
those laboratories directly involved in a material comparison
must conduct interlaboratory studies specifically aimed at the
material of interest.4
4
Following the ASTM Research Report format guide, prepare a research report
on the ILS to be filed at ASTM Headquarters.
ANNEXES
(Mandatory Information)
A1.1 Underlying Assumptions of ILS the specified repeatability conditions. This assumption is not
A1.1.1 Within-Laboratory Variability—The cell standard always fulfilled. However, the shorter the period of time in
deviation is a measure of the within-laboratory variability of which the test results for a particular material are to be obtained
each individual laboratory. All laboratories are assumed to by the laboratories the more likely the validity of this assump-
have essentially the same level of variability when following tion. Therefore, the laboratory cell variances can generally be
sR 5 Œ s x̄2 1
~ n 2 1 ! s r2
n
(A1.4)
laboratories combined. Values of k larger than 1 indicate
greater within-laboratory variability than the average for all
laboratories. Since such variation among laboratories is
When sR calculates to less than sr, sR is set equal to sr. expected, critical values of k have been calculated to aid in the
decision of whether the cell standard deviation of one labora-
A1.2 Consistency Statistics
tory is sufficiently different from the rest of the laboratories as
A1.2.1 Critical Values—The derivation of the equations for to require investigation.
calculating critical values of h and k are given in A1.2.2 and A1.2.3.2 A valid test for determining whether a particular
A1.2.3. In each case critical values were calculated at three cell variance is inconsistent relative to the variances of the
significance levels, 1 %, 0.5 %, and 0.1 %. Of these three only other laboratories is to calculate the F-ratio of the one cell
the 0.5 % critical values were chosen for flagging as described variance to the pooled variance of all the other laboratories—
in Section 17. This choice is based on the judgment from excluding the variance being tested. This is shown in Eq A1.10
experience that the 1 % values are too sensitive (flag too many) as follows:
!
p sL = estimated between-laboratory standard deviation,
k5 (A1.13)
p21 sr = estimated within-laboratory standard deviation, and
11
F nj = number of measurement results in cell j.
A1.2.3.3 The upper critical value of k is calculated by Eq A1.3.2.3 In the case of a balanced data set, where all the cell
A1.13 where the F value is the upper 99.5th percentile of the F sizes nj are equal, all the weighting factors wj are also equal and
distribution with n – 1 and (p – 1) (n – 1) degrees of freedom can be replaced by 1, in which case the following equations are
for selected combinations of number of within-laboratory test obtained:
results (n) and number of laboratories (p). The values obtained x̂ * 5 x% * (A1.16)
are given in Table 5. The spreadsheet function for such values *
SS ˜
of F is FINV(0.005,(n-1),(n-1)*(p-1)). d
5 s x̄*2 (A1.17)
p22
Œ
A1.3 Consistency Statistics for Unbalanced Data Sets
p21
A1.3.1 The derivations of the consistency statistics, h and k, ζ 5 ~ x̄ c 2 x% * ! (A1.18)
p
presented above in Section A1.2 presume a balanced data set.
Derivations of h and k for unbalanced data sets are given below where:
in A1.3.2 and A1.3.3. The definitions of h and k given here x% * = average of all the cell averages except the one being
reduce to those given in Section A1.2 when the data set is compared, and
balanced. s x̄* = standard deviation of all the cell averages except the one
being compared.
A1.3.2 Between-Laboratory Consistency:
A1.3.2.1 As in A1.2.2, the critical values for the between- The equation for t in this case, Eq A1.14, is equivalent to Eq
laboratory consistency statistic, h, are based on its relationship A1.5.
to a t-statistic, which in this case is given by Eq A1.14: A1.3.2.4 Additional equations are required in order to
express h in terms of the t-statistic from Eq A1.14:
ζ
t5 (A1.14) p
=SS *
˜
d
⁄ ~p 2 2! ( ~w
j51
j x̄ j !
x̂ 5 p (A1.19)
where:
(
j51
wj
t = observed Student’s t value,
S
x̂ 2 x̄ c w c ⁄ ( wD
p
j
For a balanced data set, where all the cell sizes nj are equal,
S ( wD
x̂ * 5
j51
p (A1.21) Eq A1.27 is equivalent to Eq A1.10.
1 2 wc ⁄ A1.3.3.2 The repeatability variance s r2 is defined to be the
j
j51 pooled variance of all the cells — including cell c — and is
x̄ c 2 x̂ given explicitly by Eq A1.28:
ζ5 (A1.22)
!
p
1 1 1
wc
2 p s r2 5
N 2 p j51 j(
~ n 2 1 ! s j2 (A1.28)
(w
j51
j
where:
*
SS 5 SS ˜d 2 ζ 2
˜
d
(A1.23) N = total number of test results,
p = number of laboratories,
For a balanced data set, with all weighting factors wj set
nj = number of test results in cell j, and
equal to 1, Eq A1.21 and Eq A1.23 are analogous to Eq A1.6 s j2 = cell variance of cell j.
and Eq A1.7, respectively.
A1.3.2.6 When Eq A1.23 is substituted in Eq A1.14, the Then the pooled variance s P2 without cell c is given by Eq
result is the following: A1.29:
ζ ~ N 2 p ! s r2 2 ~ n c 2 1 ! s 2c
t5 s P2 5 (A1.29)
(A1.24) ~N 2 p! 2 ~nc 2 1!
=~ SS ˜
d 2 ζ 2! ⁄ ~ p 2 2 !
Defining pc = (N − p) ⁄ (nc − 1) leads to Eq A1.30 below:
A1.3.2.7 The h-statistic is defined as follows:
Œ
p c s r2 2 s 2c
s P2 5 (A1.30)
ζ p21 pc 2 1
h5 (A1.25)
=SS ˜
d ⁄ ~p 2 1! p
A1.3.3.3 The consistency statistic k is defined by Eq A1.31:
~ p 2 1 !~ x̄ c 2 x̂ ! sc
!S ( D
5 k5 (A1.31)
sr
1 1
wc
2 p ~ p ! ~ SS ˜d ! Combining Eq A1.27, Eq A1.30, and Eq A1.31 results in Eq
wj A1.32:
j51
!
pc
~p 2 1!t k5 (A1.33)
h5 (A1.26) pc 2 1
=p ~ t 2 1 p 2 2 ! 11
F
Since Eq A1.26 is equivalent to Eq A1.9, the same critical A1.3.3.5 The upper critical value of k is given by Eq A1.33
values of h calculated for use with balanced data sets are with F equal to the upper 99.5th percentile of the F-distribution
appropriate for use with unbalanced data sets. Critical values of with nc − 1 and (N − p) − (nc − 1) degrees of freedom. The
h are presented in Table 5. spreadsheet function for this percentile of F is FINV(0.005,nc-
A1.3.3 Within-Laboratory Consistency: 1,(N-p)-(nc-1)).
A1.3.3.1 As in A1.2.3, the test for determining whether a A1.3.3.6 If the data set is balanced, with all the nj equal to
particular cell variance is inconsistent with the variances of the n, then N = np, nc = n, pc = p, and Eq A1.33 is equivalent to Eq
other laboratories is based on the F-ratio of the one cell A1.13. Critical values of k for this case are given in Table 5.
variance to the pooled variance of all the other laboratories — Note that the same tabulated critical values of k must not be
excluding the one being tested. However, in the case of an used with unbalanced data sets.
A2.1 The protocol for an ILS is designed so that each lab A2.3.1 Data Entry—Enter the test result data (xij values) for
conducts an equal number of replicate test results for a one material (from one column of Table 1) into the worksheet.
material, resulting in a balanced data set. At times, there may The number of replicates (ni) for each laboratory will also be
be missing values due to labs submitting less than the requested entered into an adjacent column either manually or by using the
number of replicates, thus giving rise to an unbalanced data worksheet Count function. Note that there is a missing value
set. The calculations in Section 15 of this standard are based on for Replicate 2 in Laboratory 4. The calculations for the
a balanced data set. This annex shows how the calculations are unbalanced data set precision estimates are shown in the
carried out for an unbalanced data set. following Sections A2.4 – A2.6. The calculations for the
A2.1.1 This annex deals with the calculations for precision consistency statistics are shown in Section A2.7.
statistics and consistency statistics for an unbalanced data set A2.4 Laboratory Statistics
by allowing for different numbers of replicate test results from
the participating laboratories. A2.4.1 Laboratory Averages, x̄ i —Calculate the average for
each laboratory using the following equation:
A2.1.2 Example—The calculations in this annex are again ni
illustrated with test results from the ILS in which the concen-
tration of glucose in serum was measured at five different
x̄ i 5 (x
j51
ij ⁄ ni (A2.1)
!
of an unbalanced data set in this annex. ni
( ~x
j51
ij 2 x̄ i ! 2
A2.2 For an unbalanced data set, the symbols and equations si 5 (A2.2)
are modified by placing subscripts on the symbols for the test ~ni 2 1!
result data, using the letter i for the laboratory number and the If ni = 1, si is set to zero.
letter j for the replicate number within the laboratory. Thus, the The symbols have the same meaning as for Eq A2.1. Thus,
second test result in Laboratory 4 would be designated with the for Laboratory 1:
Œ Œ
symbol x42. (If the number of laboratories exceeds 9, then the
notation x10,3 can be used.) Additionally, a new symbol ni is @ ~ 20.537! 2 1 ~ 0.633! 2 1 ~ 20.097! 2 # 0.698467
s1 5 5
introduced for the number of replicates in Laboratory i. ~3 2 1! 2
5 0.591
A2.3 Worksheets—As in Section 15, the calculations are
facilitated by a separate calculation worksheet for each A2.5 Intermediate Statistics
material, using Table 2 as a model for each material, but
making appropriate changes for different numbers of test A2.5.1 Total Number of Data, N—Calculate the total num-
results per laboratory as shown in Table A2.1. Statistics for ber of test result data from all p laboratories using the
each laboratory are placed in columns adjacent to their data following equation:
p
columns, and statistics for the entire data set are listed below
the rows of laboratory data and statistics. N5 (n
i51
i (A2.3)
where:
p = the number of laboratories in the data set,
N = the total number of test result data for the material, and
ni = the number of test results conducted by Laboratory i.
Thus, for Material C with missing value in Laboratory 4:
N 5 313131213131313 5 23
A2.5.2 Grand Average, x% —Calculate the grand average as
FIG. A2.1 Dot Plot of Submitted Test Results for Material C the weighted average of all the laboratory averages, weighted
where:
x ij = individual test result (see A2.3),
x̄ i = laboratory average (see A2.4.1),
si = laboratory standard deviation (see A2.4.2),
% =
x average of laboratory averages (see A2.5.2),
di = laboratory deviation (see A2.5.3),
s x̄ = standard deviation of laboratory averages (see A2.5.5),
sr = repeatability standard deviation (see A2.6.1),
sL = between-laboratory standard deviation (see A2.6.2), and
sR = reproducibility standard deviation (see A2.6.3).
i51
n i2 ⁄N ⁄ p 2 1! (A2.6)
x% 5 ( n x̄ ⁄ N (A2.4)
F S DG
i i
i51
67
23 2
where: 23
n* 5 5 2.87
x% = the grand average, ~8 2 1!
x̄ i = the individual laboratory averages, and For moderate numbers of missing values, n* will usually be
N = the number of data in the data set. close, but not exactly equal, to the average number of replicates
Thus, for Material C with missing value: per lab.
3 ~ 133.197! 1...12 ~ 137.095! 1...13 ~ 134.743! 3095.13 A2.5.5 Standard Deviation of the Laboratory Averages,
x% 5 5 s x̄ —Calculate this statistic using the following equation:
N 23
5 134.5709
!
p
Œ Œ
d i 5 x̄ i 2 x% (A2.5)
3 ~ 21.374! 2 1...13 ~ 0.172! 2 51.2009
Thus, for Laboratory 1: s x̄ 5 5 5 =2.54896
2.87~ 8 2 1 ! 2.87~ 7 !
d 1 5 133.197 2 134.5709 5 21.374 5 1.5965
Note that a laboratory submitting only one test result can Note that a laboratory having only one test result can
have a laboratory deviation because its average is the single contribute to an estimate of laboratory variation although it
test result. only has a small weight.
A2.5.4 Operational Number of Replicates, n*—For Mate- A2.6 Precision Statistics
rial C, calculate the operational number of replicates5 using the
A2.6.1 Repeatability Standard Deviation, sr—Calculate this
following equation:
statistic using the following equation:
!
p
5
Searle, S. R., Casella, G., and McCulloch, C., E., Variance Components, John
( ~n
i51
i 2 1 ! s i2
sr 5 (A2.8)
Wiley & Sons, Inc., New York, 1992. ~N 2 p!
TABLE A2.2 Interlaboratory Study Worksheet for Consistency of Glucose in Serum Unbalanced Data for Material C
Test Results, x ij Cell Statistics
Laboratory
1 2 3 ni x̄ i si wi d̃ i hi ki kC,i
1 132.66 133.83 133.10 3 133.197 0.591 0.39819 −1.436 −0.89 0.38 2.04
2 132.92 136.90 136.40 3 135.407 2.168 0.39819 0.774 0.48 1.38 2.04
3 132.61 135.80 135.36 3 134.590 1.729 0.39819 −0.043 −0.03 1.10 2.04
4 138.50 ... 135.69 2 137.095 1.987 0.34198 2.462 1.40 1.26 2.57
5 131.90 134.14 133.76 3 133.267 1.199 0.39819 −1.366 −0.85 0.76 2.04
6 137.21 135.14 137.50 3 136.617 1.287 0.39819 1.984 1.23 0.82 2.04
7 130.97 131.59 134.92 3 132.493 2.124 0.39819 −2.140 −1.33 1.35 2.04
8 135.46 135.14 133.63 3 134.743 0.977 0.39819 0.110 0.07 0.62 2.04
where:
x ij = individual test result (see A2.3),
x̄ i = laboratory average (see A2.4.1),
s i = laboratory standard deviation (see A2.4.2),
x̂ = weighted grand average (see A2.7.2.2),
w i = cell weighting factor (see A2.7.2.1),
d̃ i = laboratory deviation from the weighted grand average (see A2.7.2.3),
SS d˜ = weighted sum of squares of laboratory deviations (see A2.7.2.4),
s r = repeatability standard deviation (see A2.6.1),
s L = between-laboratory standard deviation (see A2.6.2),
s R = reproducibility standard deviation (see A2.6.3),
h i = between-laboratory consistency (see A2.7.2.5),
k i = within-laboratory consistency (see A2.7.3.1), and
k C,i = critical value for ki (see A2.7.3.2).
ŒS D
hi 5 (A2.16)
wi = the weighting factor for the ith cell, 1 1
sL = the estimated between-laboratory standard deviation wi
2
Σw j
×SS d˜ × p
(see Eq A2.10),
sr = the estimated within-laboratory standard deviation (see where:
Eq A2.8), and hi = the between-laboratory consistency statistic for Labo-
ni = the number of test results from Laboratory i. ratory i,
Thus, for the unbalanced data set of Table A2.2: d̃ i = the cell deviation for Laboratory i,
p = the number of laboratories in the data set,
1 wi = the weighting factor for the ith cell,
w4 5 5 0.34198
1.29842 1 ~ 1.57372 ⁄ 2 ! Σw j = the sum of all the weighting factors, and
SS ˜d = the weighted sum of squares of deviations.
and:
1
Retain two decimal places in the computed values of hi. To
wi 5 5 0.39819, for ifi4 assess data consistency, compare the absolute value | h i | to the
1.2984 1 ~ 1.57372 ⁄ 3 !
2
appropriate critical value from Table 5 for the number of
A2.7.2.2 Weighted Grand Average, x̂ —Calculate the laboratories, p.
weighted grand average using Eq A2.13.
Thus, for Laboratory 4:
p p
x̂ 5 ( w x̄ ⁄ ( w (A2.13) 2.462 × ~ 8 2 1 !
ŒS
i i i
D
i51 i51 h4 5 5 1.40
1 1
where: 2 ×7.272 × 8
0.34198 3.12931
x̂ = the weighted grand average,
wi = the cell weighting factors, Table 5 gives the critical value 2.15 for h (8 laboratories).
x̄ i = the individual laboratory averages calculated by Eq Since | 1.40 | < 2.15, there is no apparent data inconsistency.
A2.1, and A2.7.3 Compute the k Statistics—The within-laboratory
p = the number of laboratories. consistency statistic, k, checks the consistency of the cell
In general, x̂ will differ from the grand average x% calculated variances for different laboratories. Since these variances
by Eq A2.4, since different weighting factors are used. represent only within-laboratory variance, which under the null
Thus, for the unbalanced data set: hypothesis is the same for all laboratories, the methodology for
k-statistics with unequal cell sizes does not depend on how the
x̂5
total variance is partitioned into between-laboratory and
~ 0.39819!~ 133.197! 1...1 ~ 0.34198!~ 137.095! 1...1 ~ 0.39819!~ 134.743! within-laboratory components.
0.398191...10.341981...10.39819
A2.7.3.1 For each laboratory, calculate a value of ki using
421.307
5 5 134.633 Eq A2.17.
3.12931
si
A2.7.2.3 Laboratory Deviation, d̃ i —Recalculate each labo- ki 5 (A2.17)
sr
ratory deviation by subtracting the weighted grand average, x̂ ,
from the laboratory average using Eq A2.14: where:
si = the cell standard deviation calculated by Eq A2.2, and
d̃ i 5 x̄ i 2 x̂ (A2.14) sr = the repeatability standard deviation calculated by Eq
In general, d̃ i will differ from the laboratory deviation di A2.8.
calculated by Eq A2.5. Retain two decimal places in the computed values of ki.
Thus, for Laboratory 4 of the unbalanced data set: These are listed in Table A2.2.
Thus, for Laboratory 4:
d̃ 4 5 137.095 2 134.633 5 2.462
s4 1.987
k4 5 5 5 1.26
A2.7.2.4 Weighted Sum of Squares of Deviations, s r 1.5737
SS ˜d —Calculate the weighted sum of squares of the laboratory
A2.7.3.2 Critical Values of k—Do not use Table 5 for
deviations using Eq A2.15.
critical values of ki. Instead, calculate a critical value for each
p
ki using Eq A2.18.
SS d˜ 5 ( w d̃
i51
i
2
i (A2.15)
!
pi
Thus, for the unbalanced data set: k C,i 5 (A2.18)
pi 2 1
11
SS d˜ 5 0.39819 × ~ 21.436! 2 1...10.34198 × ~ 2.462! 2 1... F
10.39819 × ~ 0.110! 2 5 7.272 where:
A2.7.2.5 For each laboratory, calculate a value of h using Eq kC,i = the critical value for ki,
pi = (N − p) / (ni − 1),
A2.16:
F = F0.995(ni − 1, N − p − ni + 1), the 99.5th percentile of Since 1.26 ≤ 2.57, k4 does not exceed the critical value,
the F-distribution with ni − 1 and N − p − ni + 1 indicating no issue with data inconsistency.
degrees of freedom. A2.7.4 Data Consistency Evaluation—Table A2.2 contains
Compare the value of ki to kC,i to decide whether the cell the data and calculations of the h and k consistency statistics
standard deviation for Laboratory i is inconsistent with the rest. for the glucose in serum example. Table 5 gives the (same)
Thus, for Laboratory 4: critical value 2.15 for each hi (8 laboratories). The critical
23 2 8 value for each ki is given in the last column of Table A2.2.
p4 5 5 15
221 None of the hi statistics were outside the two-sided range
F 5 F 0.995~ 2 2 1, 23 2 8 2 2 1 1 ! 5 F 0.995~ 1, 14! 5 11.060 0 6 2.15, and none of the ki statistics exceeded their respective
critical values, kC,i, indicating no issues with data
! !
p4 15 inconsistency.
k C,4 5 5 5 =6.6202 5 2.57
p4 2 1 15 2 1
11 11
F 11.060
APPENDIX
(Nonmandatory Information)
X1.1 ILS Calculations Using a Spreadsheet X1.1.2.1 Different numbers of laboratories and replicates
X1.1.1 A spreadsheet for performing Practice E691 calcu- per laboratory are accommodated by expanding or contracting
lations is shown in Fig. X1.1 and uses the glucose in serum the area of the test result entry. This should be done by
Material C example in Section 15. This ILS had eight labora- inserting or deleting the middle rows and columns of the data
tories with each laboratory conducting three replicate test entry area to preserve the calculation formulas.
results on each of five materials. X1.1.3 Calculation of Cell Statistics—Five cell statistics
X1.1.2 Data Entry—Test results are entered in cells B3 – are calculated for each laboratory and appear in Columns
D10, one row for each lab. The number of laboratories is E – I for this example. The symbols, columns, and spread-
entered in B12 and the number of replicates per laboratory is sheet formulas for Laboratory 1 are listed in Table X1.1 below.
entered in B13, as these values are used in subsequent For the other laboratories these formulas can be dragged down
calculations. through the rows.
X1.1.4 Calculation of Statistics for the Material—These • E16 Reproducibility standard deviation, sR
calculations are made in Cells E12 – E16 in the spreadsheet
X1.1.5 Disclaimer—This spreadsheet example is not sup-
example below, and comprise the following:
• E12 Average of cell averages, x% ported by ASTM, and the user of this standard is responsible
• E13 Standard deviation of the cell averages, s x̄ for its use. For questions pertaining to use of this spreadsheet
• E14 Repeatability standard deviation, sr example, please contact Subcommittee E11.20.
• E15 Between-laboratory standard deviation, sL
ASTM International takes no position respecting the validity of any patent rights asserted in connection with any item mentioned
in this standard. Users of this standard are expressly advised that determination of the validity of any such patent rights, and the risk
of infringement of such rights, are entirely their own responsibility.
This standard is subject to revision at any time by the responsible technical committee and must be reviewed every five years and
if not revised, either reapproved or withdrawn. Your comments are invited either for revision of this standard or for additional standards
and should be addressed to ASTM International Headquarters. Your comments will receive careful consideration at a meeting of the
responsible technical committee, which you may attend. If you feel that your comments have not received a fair hearing you should
make your views known to the ASTM Committee on Standards, at the address shown below.
This standard is copyrighted by ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959,
United States. Individual reprints (single or multiple copies) of this standard may be obtained by contacting ASTM at the above
address or at 610-832-9585 (phone), 610-832-9555 (fax), or service@astm.org (e-mail); or through the ASTM website
(www.astm.org). Permission rights to photocopy the standard may also be secured from the Copyright Clearance Center, 222
Rosewood Drive, Danvers, MA 01923, Tel: (978) 646-2600; http://www.copyright.com/
Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.
Alternative Proxies: