5

Reliability of Measurements

In pure mathematics or in counting, every number has an exact meaning. The

figure 2, for example, means precisely two units (not approximately two units).
In using numbers to express the results of measurements, however, we use
numbers with inexact meanings, because no measurement is perfectly accurate.
If we say that an object has a length of 2 m, we mean that it is approximately 2
m long. We would not be surprised to find that the actual length differs from
2.000000 m.
In expressing the results of measurements, we should use numbers in a way
that indicates the reliability of the result. Suppose two persons measure the
diameter of a dime with a centimeter scale. One person reports a result of 1.79
cm; the other finds the diameter to be 1.80 cm. Both would agree that the
desired reading is near the 1.8 cm mark and just slightly toward the 1.7 cm side
of that mark. However, one person has estimated the value as 1.79 cm, whereas
the other feels that the edge of the dime is close enough to the mark to report a
value of 1.80 cm. How should the uncertainty be expressed in reporting this
result? It would be correct to report the result either as 1.79 cm or as 1.80 cm,
but it would not be correct to give the average value of 1.795 cm. This last figure
implies that the true value is known to lie near the middle of the range between
1.79 cm and 1.80 cm, which is not the case.
Suppose the measurements are repeated with a caliper whose vernier scale
permits careful measurement to the nearest 0.01 cm and estimation to the
44 Reliability of Measurements

nearest 0.001 cm. Now the two results might be 1.792 cm and 1.796 cm. It
would now be proper to report the average value of 1.794 cm, because the third
decimal place is reasonably well known. It would be even more informative to
report the result as 1.794 ± 0.002 cm. The symbol ± is read as "plus or
minus." It shows that the actual results vary by 0.002 cm in either direction
from the reported average value.
In talking about the results of measurements, we distinguish between the
accuracy and the precision of the results. The accuracy of a series of measure-
ments tells how closely the average of the results agrees with the true value of
the quantity that is measured. The precision of a series of measurements tells
how nearly the repeated measurements yield the same result. For example,
suppose that the markings on a centimeter scale are placed too far apart (as if
the scale has been stretched). In this case, the results obtained in a series of
measurements of the same object might be quite precise (different measure-
ments would yield nearly the same answer), but they would be inaccurate (the
average result would be far from the true value).
Measurements commonly involve systematic errors. These are errors that are
reproducibly introduced in each measurement because of the construction, use,
or calibration of the equipment (as in the case of the stretched scale). The
precision of the results may give the illusion of accuracy in such cases. For this
reason, it is desirable to make a measurement by various entirely different
methods. If the results still show high precision (close agreement with one
another), then it is unlikely that systematic errors exist. The accuracy of the
measurement can also be tested by using the same measurement methods on a
"standard sample" whose value has been certified by some reliable institution,
such as the National Bureau of Standards.
Measurements also commonly involve random errors. These are errors whose
size and direction differ from measurement to measurement; that is, they are
unpredictable and unreproducible. They are commonly associated with the
limited sensitivity of instruments, the quality of the scales being read, the
degree of control over the environment (temperature, vibration, humidity, and
so on), or human frailties (limitations of eyesight, hearing, judgment, and so
on). We shall say much more about random error later in this chapter.

SIGNIFICANT FIGURES

All digits of a number that are reasonably reliable are known as significant
figures. The number 1.79 has three significant figures: 1, 7, and 9. The number
1.794 has four significant figures.
The position of the decimal point in a measured value has nothing to do with
the number of significant figures. The diameter of a dime may be given as 1.794
cm or as 17.94 mm. In either case, four significant figures are used.
Significant Figures 45

PROBLEM:
A student weighs a beaker on a triple-beam balance, finding values of 50.32 g,
50.31 g, and 50.31 g in successive weighings. Express the average weight to the
proper number of significant figures.
SOLUTION:
To obtain the average, add the weights and divide by 3.

50.32 g
50.31 g
50.31 g
Total = 150.94g
Average = 50.3133 g

Because the weighings disagree in the second decimal place, it is not proper to
give the weight to more than the second decimal place. The average weight, to the
proper number of significant figures, is 50.31 g.

Final Zero as Significant Figure

Final zeros after the decimal point are significant figures and are used to
indicate the decimal place to which the measurements are reliable. Thus 1.0 cm
indicates a length reliably known to tenths of a centimeter but not to hundredths
of a centimeter, whereas 1.000 cm indicates a length reliably known to thou-
sandths of a centimeter. A very common mistake is leaving out these zeros
when the measured quantity has an integral value.

PROBLEM:
Give the value of a 10 g weight to the proper number of significant figures. The
balance on which the weight is used will respond to weight differences of 0.0001 g.
SOLUTION:
The weight is given as 10.0000 g, to show that it is reliable to 0.0001 g.

When a number has no zeros after the decimal point, final zeros before the
decimal point may or may not be significant, depending upon the usage. If we
say that there are 1000 students enrolled in a school, all the zeros probably are
significant. But if the population of a city is given as 360,000, the last two or
three zeros are not significant, because daily changes make the population
uncertain by perhaps several hundred persons. The final zeros in this case are
used only to indicate the position of the decimal point. A convenient way to
indicate reliability of a number that has final zeros before the decimal point is to
Significant Figures 47

Significant Figures in Results of Calculations

The results of measurements often are used to calculate some other result. In
such a case, the result of the calculation should be expressed with an appropri-
ate number of significant figures to reflect the reliability of the original mea-
surements. There are two rules for this procedure.
1. Addition or subtraction. The values to be added or subtracted should all be
expressed with the same units. If they are expressed in scientific notation, they
should all be expressed with the same power of 10. The result should be
rounded off so that it has only as many digits after the decimal point as the
number with the fewest digits after the decimal. For example, consider the sum
of the following weights:

13.8426 g
764.5 g
7.08 g
Sum = 785.4226 g

This mathematical sum is a very misleading statement because it contains seven

significant figures. It implies that the total weight is known to the nearest 0.0001 g
when, in fact, one of the weights being added is known only to the nearest 0.1 g
and another only to the nearest 0.01 g. The weight known least reliably (to the
fewest decimal places) limits the reliability of the sum. Therefore, the sum is
properly expressed as 785.4 g. (Note that the number of significant figures in the
weights is irrelevant in applying this rule for addition or subtraction. The proper
result has four significant figures, although one of the weights being added has
only three significant figures. It is the number with the fewest digits after the
decimal point that determines the position of the least reliable digit in the
answer.)
2. Multiplication or division. The product or quotient should be rounded off to
the same number of significant figures as the least accurate number involved in
the calculation. Thus, 0.00296 x 5845 = 17.3, but 0.002960 x 5845 = 17.30.
However, this rule should be applied with some discretion. For example, con-
sider the following multiplication:

0.00296 x 5845 x 93

The rule indicates that the result should be rounded off to two significant
figures, so that the product would be 1600, or 1.6 x 103. However, an error of
± 1 in the value of 93 is not much more significant than an error of ± 1 in 102; we
can say that 93 almost has three significant figures. Because the other numbers
involved all have at least three significant figures, it would be reasonable to
report the result of this multiplication as 1.61 x 103 (using three significant
figures). Obviously, such decisions must be made by common sense rather than
48 Reliability of Measurements

a hard and fast rule. The best situation would be to have a range of error for
each number; then the calculation could be performed with all the largest or all
the smallest values to see which digits in the result are reliable.
A case can be made for rounding off all numbers involved in a calculation
before the calculation is actually made. However, in this age of hand cal-
culators this procedure adds little except extra effort. The simplest approach
probably is to enter all the numbers of a calculation through the keyboard with-
out regard to significant figures. Then, by inspection at the end of the calcula-
tion, determine how many decimal places should be used (for addition and
subtraction) or how many significant figures should be used (for multiplication
and division) in the final result. The calculator cannot make the decision about the
proper number of decimal places or significant figures to use; only the operator can
do this. One of the most common errors in the use of calculators is to write
down all of the digits that appear in the display as a result of a calculation,
regardless of their significance. You must learn to think about your answers.
A pure number such as 3 or 4 has an unlimited number of significant figures
(4.000000 ), as does a defined quantity such as TT (3.14159...) or e
(2.7182818 . . . . ) . Do not fall into the trap of excessively rounding off results that
come from equations using pure or derived numbers. For example, if you want
to find the volume of a sphere whose radius has been measured as 15.13 cm,
you shouldn't round off the answer to 1 x 104 cm 3 just because you are going to
use the formula V = %m-3, in which 4, 3, and rr each appears to have only one
significant figure. It is the measured values that determine the number of sig-
nificant figures. In this case, the volume should be expressed to four significant
figures as 1.451 x 104 cm3. Your calculator probably has a TT key that will give
with one stroke the value of TT to 8 or 10 decimals.

DISTRIBUTION OF ERRORS

We have talked about significant figures and the general unreliability of the
"last figure" of a measurement. Now we shall talk about just how unreliable
these last figures are. Your experience has shown that really gross errors rarely
occur in a series of measurements. Suppose you were able to make an infinite
number of measurements on the same quantity (call it x). You would not be
surprised if, on plotting each observed value of x against the frequency with
which it occurred, you obtained a symmetrical curve similar to that shown in
Figure 5-1. One of the advantages of making an infinite number of measure-
ments is that the average (x) of the values will be equal to the "true value" (/A),
represented by the dotted vertical line drawn from the peak of the curve. As
expected, the more a value of x deviates from p, the less frequently it occurs.
This curve is symmetrical because there is equal probability for + and - errors;
it is called a normal distribution curve. If you made your measurements in a more
careless manner or with a less sensitive measuring device, you would obtain a
distribution curve more like that in Figure 5-2, shorter and broader but with the
 49

FIGURE 5-1
Normal distribution curve with a small standard deviation.

same general shape. The breadth of a distribution curve (the spread of results)
is a measure of the reliability of the results. Two major ways of describing this
spread are average deviation and standard deviation.

Average Deviation

The deviation of each individual measurement (x) from the average (x) of all the
measurements is found by simple subtraction; the deviation of the /th mea-
surement is xt - x. We are interested in the sizes of the deviations, without
regard to whether they are + or -. That is, we are interested only in the
absolute values of the deviation, . The average deviation is simply the
average of these absolute values:

x, - x x2 — x
Average deviation =

FIGURE 5-2
Normal distribution curve with a large standard deviation.
50 Reliability of Measurements

Simple expressions for x and average deviation can be made by using the
symbol 2, which means "the sum of" whatever follows it:

and

SU - x
Average deviation = —Lj - L (5-1)

In each case, the sum of n values is understood. One important point to re-
member: x may be calculated for any number of measurements, but only for an
infinite number of measurements will x = p, the "true value."

PROBLEM:
Five persons measure the length of a room, getting values of 10.325 m, 10.320 m,
10.315 m, 10.313 m, and 10.327 m. Find the average value and the average devia-
tion.
SOLUTION:
Add the separate values and divide by 5 to get the arithmetical mean. Set opposite
each value its deviation from the average, without regard to sign. Take the average
of these deviations.
Measurement Deviation
xt xt — x
10.325 m 0.005 m
10.320 0.000
10.315 0.005
10.313 0.007
10.327 0.007
SJT, = 51.600 m 0.024 m = 2|jr, - x\

^ = 10.320 m ^ ' ~ *' = 0.0048 m = Average deviation

The average is 10.320 m, with an average deviation of 0.0048 m. It is proper

to write the average as 10.320 m, because the deviation affects digits in only the
third decimal place. It is not correct to give the length as 10.32 m; this implies
that the measurement is uncertain in the second decimal place. Average devia-
tion is one of the simplest measures of reliability of measurements (the spread
of experimental values), but a better estimate of reliability can be made with
standard deviation.
Distribution of Errors 51

Standard Deviation
The standard deviation is the square root of the variance (s2). The variance is
almost the same as the average of the squares of the deviations of the measure-
ments from the average (x); it is defined as

variance = ,» = (*' " ^ + <*• ~ ^ + (x* ~ *>' + ' ' ' + (x

»~&
n- 1

(5-2)
n- 1

For reasons that we need not discuss here, n - 1 is used as the denominator
instead of n. Of course, for very large values of n (say, 1000), there is no
appreciable difference between n and n - I . Thus, for very large numbers of
measurements, you really can say that the variance is the average of the
squares of the deviations. Because the standard deviation is the square root of
the variance, we have

standard deviation = s = \; ———-r— (5-3)

V n - I
The standard deviation will have the same units as the original measurements,
and the same units (but not the same value) as the average deviation.

PROBLEM:
Calculate the standard deviation (s) of the measurements made in the preceding
problem.
SOLUTION:
As in the last problem, first find the average of the measurements, and then
subtract it from each of the individual measurements to get the deviation. The sum
of the squares of these deviations divided by n - 1 is the variance.
Measurement Deviation (Deviation)'2
xt (xt - x) (xt - x)'2
10.325 m 0.005 m 0.000025 m 2
10.320 0.000 0.000000
10.315 -0.005 0.000025
10.313 -0.007 0.000049
10.327 0.007 0.000049
= 51.600m 0.000148 m 2 =
= 10.320m
:
Standard deviation = s = \ —=
V n - 1
52 Reliability of Measurements

In order to make the value of s useful for other calculations, it is customary to

write it to one more decimal place than the least significant figure of the mea-
surements; we shall do the same for x when it is used in the calculation ofs, but at
no other time. If we did not adopt such a convention, we would frequently find,
for a good set of data, that a standard deviation rounded off to the decimal place
corresponding to the least significant figure of the measurements would have a
value of 0. This would be a useless and misleading result because variations in
the measurements actually exist. A good compromise is to write the average to
the correct number of significant figures as discussed on p 46, but to write the
error statement to one more decimal place than corresponds to the least signifi-
cant figure in the average. An error statement is the average deviation, the
standard deviation, or one of the various confidence intervals to be discussed on
pp 56-57. Thus, in the example just shown, you would write
x = 10.320 ± 0.0048 m (for average deviation)
x = 10.320 ± 0.0061 m (for standard deviation)
For a very poor set of data, with a very large standard deviation, it is foolish to
show the error statements to one more decimal place than the least significant
figure. If the room measurements just cited actually had a standard deviation of
±0.0756 m, it means that you were silly to think of measuring to the nearest
0.001 m, and in any case you should round the standard deviation to ±0.076 m.
The decision as to when you do, or do not, write the error statement to one
more decimal place than the least significant figure is arbitrary. (Some profes-
sors say that you should use the extra decimal place if the standard deviation is
less than 0.4% of the average of the measurements.)
Many calculators permit you to determine x and j directly without the need of
setting out the calculations as described in the last problem. After entering each
measurement through the keyboard, you press the S+ key. After entering all of
the numbers in this fashion, you press the x key (or keys) to get the average and
then the ^ key (or keys) to get the standard deviation. The average—which may
be displayed to, say, eight decimal places—must of course be rounded off to the
proper number of significant figures.
A small complication may arise from the use of calculators that determine x
and s directly. With these calculators, the standard deviation is calculated from
the average deviation that includes all of the digits in the display. The resulting
value probably will differ slightly from the one you would obtain if you previ-
ously rounded the average to one more decimal place than the least significant
figure. This difference usually is small and can be ignored. Thus you need not
worry about somewhat different answers obtained with different types of
calculators.

Probability Distribution for Large Numbers of Measurements

In order to appreciate the usefulness of the standard deviation as a measure of
reliability, we must take a closer look at the curves in Figures 5-1 and 5-2. The
Distribution of Errors 53

mathematical equation for these curves (when they represent a very large num-
ber of measurements) is called the Gaussian distribution equation. In one of its
forms, the frequency of occurrence, F, is given by

(5-4)
I CT(27r)*J

where cr = s (the standard deviation) when n is very large, just as pi = x when

H is very large. We use these special symbols for s and * to emphasize that
this Gaussian equation does not apply to the curves obtained with only a small
number of measurements. Note that Equation 5-4 is written with the base e
rather than the base 10 (see p 17 for a discussion of natural logarithms). We
next discuss some important characteristics of this distribution curve.
1. Size of a and the shape of the curve. At the peak of the curve, x = /j.
and

In other words, the maximum height of the Gaussian curve is de-

termined solely by the value of cr and the constant 2ir. If cr is small
because the errors are relatively small, then F is large and the curve is
tall (Figure 5-1). If cr is large because of relatively large errors, then
F is small and the curve is short (Figure 5-2). For any other value of x
( Y . // ) ^
than.*, the exponent—' j —is larger for a smaller value of cr, and
the sides of the curve thus fall off faster for small cr (as in Figure
5-1) than for large cr (as in Figure 5-2).
Significance of the area under the curve. Consider the very small black-
ened area in Figure 5-1. Its width is the infinitesimal distance dx; its
height is the value of/ 7 for the value of x we have chosen. Because dx is
infinitesimal, we can regard this area as a rectangle. The area Fdx of
the rectangle represents the number of measurements of x that lie
between* and* + d*. If we take the consecutive sum of all such small
areas from one end of the curve to the other, we have the total area
under the curve, and we have included all our measurements. Because
of the way that F is denned, the total area under the curve is 1.000,
regardless of the size of cr. All probability distributions share this
characteristic that the total area under the curve equals unity. The area
under a portion of the curve represents the number of measurements of
x lying between the limiting values of* that bound the portion. For
example, if the area under a portion of the curve is 0.200, then that
portion of the curve represents 20.0 percent of the measurements.
54 Reliability of Measurements

3. Relationship between cr and area under the curve. If we express the

values of x in terms of a, we find that the normal distribution curve
always has the same shape, regardless of the size of cr. That is, the area
under the curve for a particular multiple of cr on either side of /u, is the
same for any normal distribution curve.
Values of x between Area under this portion of the curve
(j. - <r and fj. + cr 0.683 (shaded area in Figure 5-1)
/LI - 2cr and /A + 2cr 0.954 (shaded area in Figure 5-2)
/ u - 3 c r a n d | i i + 3cr 0.997
Thus, whenever you have a very large number of measurements, the
probability is that 68.3% of them (about two-thirds) will have values
within the range /A ± cr (that is, within one standard deviation of the
average value). Similarly, 95.4% of the measurements (about 19 out of
20) probably will have values within the range /u. ± 2cr, and only 0.3%
of them (3 in 1000) are likely to have values outside the range /u, ± 3cr.
This also means that the probability is only 0.3% (3 times out of 1000
measurements) that any single measurement will yield a value differing
by more than 3 cr from the value p.
4. Confidence interval and confidence level. We have seen that 3 out of 1000
measurements probably will have values outside the range /u, ± 3cr.
The range of values obtained in a particular large set of measurements
will depend on the particular extreme values that happen to be ob-
tained. Because the normal distribution curve is so regular, it is useful
to express the average result in a form that reflects some particular
percentage of the measurements, rather than listing the particular ex-
tremes obtained. For example, you might wish to report the result as
fj, ± to; where t is chosen so that the range will include some particular
percentage of the measurements. For example, we have seen that a
choice of t = 2 will yield a range that includes 95.4% of the measure-
ments. Suppose you wish to report a range that includes 80% of the
measurements; in this case, you will need to consult a t table such as
Table 5-1 to find the appropriate value o f t . We wish to find the value of
/ corresponding to a confidence level of 80%; the confidence level is the
probability that any measurement picked at random will fall within the
range /i ± t cr. Using the bottom line of the table (representing an infi-
nite number of measurements), we see that the desired value o f / is
1.282. Therefore, we can say that there is a probability of 80% that any
random measurement will fall within the range /j. ± 1.282cr; this range
is called the confidence interval. When a result is reported with a confi-
dence interval, the corresponding confidence level should be stated to
make the range meaningful. For example, a result might be reported as
25.342 ± 0.003 with a confidence level of 80%.
 55

TABLE 5-1
The t Values for Various Sample Sizes and Confidence Levels

Sample* Confidence level (percentage)

size
(n) 50 60 70 80 90 95 99
2 1 000 1 376 1 963 3078 6314 12706 63657
3 0816 1 061 1 386 1 886 2920 4303 9925
4 0765 0.978 1 250 1 638 2353 3 182 5841
5 0741 0941 1 190 1 533 2 132 2776 4 604
6 0727 0920 1 156 1 476 2015 2571 4032
7 0718 0906 1 134 1 440 1 943 2447 3.707
8 0711 0896 1 119 1 415 1 895 2365 3499
9 0706 0889 1 108 1 397 1 860 2306 3355
10 0.703 0883 1 100 1 383 1 833 2262 3250
20 0688 0861 1 066 1 328 1 729 2093 2861
30 0683 0854 1 055 1 311 1 699 2045 2756
40 0681 0851 1 050 1 303 1 684 2021 2704
50 0680 0849 1 048 1 299 1 676 2008 2678
60 0679 0848 1 046 1 296 1 671 2000 2660
120 0677 0845 1 041 1 289 1 658 1 980 2617
00 0674 0842 1 036 1 282 1 645 1 968 2576
* Statistical manuals usually list degrees of freedom in this column with values that are equal to the sample size minus one

PROBLEM:
The average result of a set of 1000 measurements is to be reported with a confi-
dence interval representing a confidence level of 80%. The average (the true value)
is 2756, and the standard deviation is 13.0. Find the confidence interval.
SOLUTION:
The set of 1000 measurements is so large that the value of? differs negligibly from
that for an infinitely large set of measurements, so we use the bottom row of Table
5-1. Looking in the column for an 80% confidence level, we find the desired value
of? to be 1.282. Therefore, the desired confidence interval is
H ± to- = 2756 ± (1.282)(13.0) = 2756 ± 17
Either of the following two statements could be made.
1. The probability is 80% that any value taken at random from the 1000 mea-
surements will lie within the interval 2756 ± 17.
2. Of the 1000 measurements, 800 (or 80%) lie within the interval 2756 ± 17.
56 Reliability of Measurements

Probability Distributions for Small Numbers of Measurements

In many cases, it is not practical to make large numbers of measurements. In

chemical analyses, there often is only enough material to make a few measure-
ments. The desire usually is to find a true value for the measured quantity, but
the average of a small number of measurements is unlikely to represent the true
value. You can see this if you make several small sets of measurements of the
same quantity; the averages of the various groups are likely to differ somewhat
from one another. Most people have the intuitive feeling that the average will be
closer to the true value as the number of measurements increases. The formal
mathematical statements of probability theory reflect this same viewpoint. It is
possible to make some statements about the results of a small number of mea-
surements, but these statements must be made with less confidence than we
have in statements resulting from large numbers of measurements.
For this discussion we use a distribution curve as before, but this time it is not
represented by an equation as simple as the Gaussian equation. In fact, there is
not just one curve; there are many, one for each size of sample (different
number of measurements). It isn't practical to draw a different curve for each
size of sample, but we can describe the changing nature of the distribution
curves: as the size of the sample gets smaller, the corresponding distribution
curve becomes shorter and broader than the ones shown in Figure 5-1 for the
same value of s. The same statement is true for Figure 5-2, where a larger value
ofs applies. This changing nature of the distribution curve is taken into account
in the t table (Table 5-1), which can be used in place of the curves.

The Precision of a Single Measurement

The calculation of standard deviation (Equation 5-3) is the sanjg whether you
havejnany measurements or only a few; sample size affects only the selection
of the / value. For a single measurement taken at random from a small number
(n) of measurements, the confidence interval for the desired confidence level is

x ± ts (5-5)

This is a statement of the precision of a single measurement.

Note that you don't know the true value (you didn't take many measure-
ments). You must use the average (x) of your measurements as the best mea-
sure available as a substitute for the true value. Note also that the value oft you
choose will depend on the sample size (n) as well as on the confidence level you
desire. The confidence interval will get larger as the number of determinations
gets smaller, or as the confidence level increases.

The Precision of the Mean

The main objective of making a series of measurements usually is to find the
true value, and we would like to indicate the confidence we may have in the
listribution of Errors 57

iverage of our values. Because the average of many measurements is more

ikely to be correct than the average of a few, our confidence interval should get
smaller as we increase the number of measurements. This is commonly ex-
jressed quantitatively by using the standard deviation of the mean, defined as

-L-=. = standard deviation of the mean

Vn

Mote that the standard deviation of the mean decreases with the square root of
i, not the first power. Thus, making 100 measurements rather than 4 does not
mprove the precision by a factor of'T = 25, but only by a factor of V25, or 5.
Fhe useful statement we can make with the standard deviation of the mean is
the following: for a series of n measurements and a specified confidence level,
the true value ofx will lie in the interval

x ± t f 4=) <5-6)
\Vrt7

This is a statement of the precision of the mean.

PROBLEM:
The density of a liquid is measured by filling a 50 ml flask as close as possible to the
index mark and weighing. In successive trials the weight of the liquid is found to
be 45.736 g, 45.740 g, 45.705 g, and 45.720 g. For these weights calculate the
average deviation, the standard deviation, the 95% confidence interval for a single
value, and the 95% confidence interval for the mean.

SOLUTION:
Because the weighings are all for the same measured volume, we first average the
weights. Let ,v refer to the weight measurement.
Weight Deviation (Deviation)*

45. 736 g 0.0107g 0.0001 14 g-

45.740 0.0147 0.000216
45.705 0.0203 0.000412
45.720 0.0053 0.000028
IY, = 182.901 g 2|.Y( - x = 0.0510 g 2(.v, - .Y)- = 0.000770 g-
2 V
.Y) 2
x- = - i A
Average A ' >•
deviation = S.Y, - .Y
.« = \/ 2 ( . Y , -
4 4 " 3

= 45.7253 g =O.OI28g = 0.0160 g

From the ; table, the t value of 3.182 is found in the row for sample size of 4 and in
the column for 95% confidence level. The precision of a single value is therefore
58 Reliability of Measurements

45.725 ± (3.182X0.0160) = 45.725 ± 0.0509 g

There is a 95% probability that any weight value picked at random has a value
that lies within 0.0509 g of the average. The precision of the mean is given by

45.725 ± (3.182) (°'°160) = 45.725 ± 0.0255 g

V4
That is, there is a 95% probability that the true value of the weight of 50 ml of this
liquid lies within the interval 45.725 ± 0.0255 g.

Relative Error
It frequently is convenient to express the degree of error on a relative basis,
rather than on an absolute basis as above. A relative basis has the advantage of
making a statement independent of the size of the measurements that were
made. For example, the statement that a solid contains 10% silver is a relative
statement; it says that one-tenth of the solid is silver, and it is understood that a
large sample of the solid would contain more grams of silver than a small one.
Percentage is "parts per hundred," and it is found for this example by multiply-
ing the fraction of the sample that is silver (in this case, 0.1) by 100. It would be
as correct to multiply the fraction by 1000 and call it " 100 parts per thousand,"
or to multiply the fraction by 1,000,000 and call it "100,000 parts per million."
The choice of parts per hundred (percentage) or parts per thousand or million is
determined by convenience. If the fraction were very small, say 0.00005, it
would be more convenient to call it 50 parts per million than to call it 0.005 parts
per hundred or 0.005 per cent.

PROBLEM:
Express the 95% confidence level of the standard deviation of the mean obtained
in the previous problem as percentage, as parts per thousand, and as parts per
million.
SOLUTION:
The 95% confidence level of the standard deviation of the mean was found to be
±0.0255 g, where the weight itself was (on the average) 45.725 g. Infraction of
the total weight that might be error is

= 0.000558
45.725 g
The relative error thus can be written as
(0.000558X100) = 0.0558 parts per hundred = 0.0558 percent
(0.000558X10') = 0.558 parts per thousand (ppt.)
(0.000558X1011) = 558 parts per million (ppm.)
Problems A 59

Calculator Tips
Equation 5-3, which defines the standard deviation, can be rewritten in the form

The advantage of this form is that you need not find the average before proceed-
ing to find the deviations. If you use a calculator that doesn't have a built-in
program for finding .s and x directly, you can easily enter each of your mea-
surements in a way that accumulates txt in one memory storage and ^,x'\ in
another. After entering all of the data, you can use S.v, and tx'] in Equation 5-7
to get s, and furthermore you can divide ~S,xt by n to obtain x, even though you
didn't have to find it to calculate s.

EPILOGUE

In an age requiring closer looks at the factors that affect our health, safety,
environment, and life style, it will become more and more important to examine
carefully and intelligently the statistical significance of the data on which impor-
tant decisions are based. The intelligent use of your calculator can be of enor-
mous help in these kinds of evaluations. You must always resist the urge to
include far more figures in your reported results than the data justify. Calcula-
tions and computers cannot improve experimental reliability, and it is your
responsibility to round off the final answer to the proper number of significant
figures.

PROBLEMS A

1. State the number of significant figures in each of the following measurements.

(a) 374; (b) 0.0374; (c) 3074; (d) 0.0030740; (e) 3740 (f) 3.74 x 105; (g) 75
million; (h) 21 thousand; (i) 6 thousandths; (j) 2 hundredths.
2. Express the answer in each of the following calculations to the proper number
of significant figures (assume that the numbers represent measurements).
(a) 3.196 + 0.0825 + 12.32 + 0.0013
(b) 721.56 - 0.394
(c) 525.3 + 326.0 + 127.12 + 330.0
(d) 5.23 x 10~J + 6.01 x 1Q-" + 8 x K)--1 + 3.273 x 10~-
3.21 x 432 x 650
563
8.57 x IP'2 x 6.02 x 10*' x 2.543
(f)
361 x 907
60 Reliability of Measurements

4.265 x (3081)- x 8.275 x 1Q-"

(g) 0.9820 x 1.0035
6.327 x 10-' x 7.056 x IP' 7 x 9.0038 x IQ-"
(h)
6.022 x 102i x 27.00 x 10~-
3. Water analysts often report trace impurities in water as "parts per million''—
that is, parts by weight of impurity per million parts by weight of water. In an
analysis, 2.5 liters of a water sample are evaporated to a very small volume in
a platinum dish; the residue is treated with a sensitive reagent that develops a
red color, whose intensity is a measure of the amount of nickel present. The
amount of nickel present is found to be 0.41 mg. How many parts per million
of nickel were present in the original sample of water? (Assume that the
density of water is 1 g/ml.)
4. State the precision, both in parts per thousand and in percentage, with which
each of the following measurements is made.
(a) 578 with a standard deviation of 2.0
(b) 0.0578 with a standard deviation of 0.00020
(c) 5078 with a standard deviation of 2.0
(d) 0.005078 with a standard deviation of 0.000030
(e) 0.0050780 with a standard deviation of 0.00000010
(f) 5078 with a standard deviation of 50
(g) 5.78 x 10' with a standard deviation of 2.0 x 10'
5. A radioactive sample shows the following counts for one-minute intervals:
2642; 2650; 2649; 2641; 2641; 2637; 2651; 2636. Find the average deviation, the
standard deviation, and the 90% confidence interval for a single value and for
the mean.
6. A student wishes to calibrate a pipet by weighing the water it delivers. A
succession of such measurements gives the following weights: 5.013 g; 5.023
g; 5.017 g; 5.019 g; 5.010 g; 5.018 g; 5.021 g. Calculate the average deviation
and the 95% confidence interval for a single value.
7. In determining the viscosity of a liquid by measuring the time required for 5.00
ml of the liquid to pass through a capillary, a student records the following
periods: 3 min 35.2 sec; 3 min 34.8 sec; 3 min 35.5 sec; 3 min 35.6 sec; 3 min
34.9 sec; 3 mm 35.3 sec; 3 min 35.2 sec. Find the average deviation and the
70% confidence interval for a single value, expressing both as a percentage.
8. A student wishes to determine the mole weight of a gas by measuring the time
required for a given amount of the gas to escape through a pinhole. He ob-
serves the following time intervals: 97.2 sec; 96.6 sec; 96.5 sec: 97.4 sec; 97.6
sec; 97.1 sec; 96.9 sec; 96.4 sec; 97.3 sec; 97.0 sec. Find the average deviation
and the 99% confidence interval for the mean, expressing both in parts per
thousand.
9. The height (h) to which a liquid will rise in a capillary tube is determined by
the force of gravity (g), the radius (/-) of the tube, and the surface tension (y)
and density (d) of the liquid at the temperature of the experiment. The rela-
tionship is y = \hdgr. A student decides to determine the radius of a capillary
Problems B 61

by measuring the height to which water rises at 25.0°C. Several attempts yield
the following heights: 75.7 mm; 75.6 mm; 75.3 mm; 75.8 mm; 75.2 mm. Find
the average deviation of these heights and the 80% confidence interval for the
mean, expressing both in parts per thousand.
10. At 25.00°C, the density and surface tension of water are 0.997044 g/ml and
71.97 dynes/cm, respectively. What actual values for these properties should
be used with the data of Problem 9 to determine the radius of the capillary?
11. A sample of a copper alloy is to be analyzed for copper by first dissolving the
sample in acid and then plating out the copper electrolytically. The weight of
copper plated is to be measured on a balance that is sensitive to 0.1 mg. The
alloy is approximately 5% copper. What size of sample should be taken for
analysis so that the error in determining the weight of copper plated out does
not exceed one part per thousand? (Remember that two weighings are needed
in order to find the weight of copper.)

PROBLEMS B

12. State the number of significant figures in each of the following measurements,
(a) 6822; (b) 6.822 x 10-<; (c) 6.82; (d) 682; (e) 0.006820; (f) 6.82 x 106; (g)
0.0682; (h) 34 thousandths; (i) 167 million; (j) 62 hundredths.
13. Express the answer in each of the following calculations to the proper number
of significant figures (assume that the numbers represent measurements).
(a) 0.0657 + 23.77 + 5.369 + 0.0052
(b) 365.72 - 0.583
(c) 365.2 + 27.3 + 968.45 + 5.62
(d) 4.27 x 10-' + 1.05 x 10~« + 5 x \0~K + 1.234 x 10-'
, , 65.4 x 1.23 x 464

6.55 x 1Q-" x 2.045 x 7.34 x 10r'

565 x 432
5280 x (2885):! x 6.570 x 1Q-'-
(g) 4.6295 x 0.8888
96.08 x 4.712 x 10-- x 7.308 x 1Q-
(h)
6.547 x 10--7 x 6.022 x 102:i
14. Water analysts often report trace impurities in water as "parts per million"—
that is, parts by weight of impurity per million parts by weight of water. A
swimming pool whose dimensions are-20 x 50 x 9 m has 14 Ib of chlorine added
as a disinfectant. How many parts per million of chlorine are present in this
swimming pool? (Assume that the density of water is 1.00 g/ml.)
15. State the precision, in both percentage and parts per thousand, with which
each of the following measurements is made.
(a) 6822 with a standard deviation of 4.0
(b) 6.822 x 10-' with a standard deviation of 0.00040
62 Reliability of Measurements

(c) 6.82 with a standard deviation of 4.0 x 10~:i

(d) 682 with a standard deviation of 4.0
(e) 0.00682 with a standard deviation of 0.0000040
(f) 6.82 x 10" with a standard deviation of 4000
(g) 0.0682 with a standard deviation of 0.004
16. A radioactive sample shows the following counts for one-minute intervals:
3262; 3257; 3255; 3265; 3257; 3264; 3259. Calculate the average deviation and
the 60% confidence interval for the mean.
17. A student wishes to calibrate a pipet by weighing the water it delivers. A
succession of such determinations gives the following weights in grams: 4.993;
4.999; 4.991; 4.994; 4.995; 4.995. Find the average deviation, the standard
deviation, and the 70% confidence interval for a single value.
18. If the viscosity of a given liquid is known, the viscosity of another may be
determined by comparing the time required for equal volumes of the two
liquids to pass through a capillary. To do this, a student makes the following
observations of time intervals: 4 min 9.6 sec; 4 min 8.8 sec; 4 min 10.2 sec; 4
min 9.8 sec; 4 min 9.0 sec. Find the average deviation and the 90% confidence
interval for the mean, expressing both in parts per thousand.
19. The time required for a given amount of gas to effuse through a pinhole under
prescribed conditions is a measure of its molecular weight. A student making
this determination observes the following effusion times: 1 min 37.3 sec; 1 min
38.5 sec; 1 min 36.9 sec; 1 min 37.2 sec; 1 min 36.5 sec; 1 min 38.7 sec; 1 min
37.0 sec. Find the average deviation and the 99% confidence interval for a
single value, expressing both as percentages.
20. The surface tension of a liquid may be determined by measuring the height to
which it will rise in a capillary of known radius. A student makes the following
observations of capillary rise with an unusual liquid that he has just prepared
in the laboratory: 63.2 mm; 63.5 mm; 62.9 mm; 62.8 mm; 63.7 mm; 63.4 mm.
Find the average deviation of these heights and the 90% confidence interval
for the mean, expressing both in parts per thousand.
21. How accurately should the values of liquid density and capillary radius be
known if all of the figures in the measurements in Problem 20 are to be
considered significant?
22. A graduated tube arranged to deliver variable volumes of a liquid is called a
buret. If a buret can be read to the nearest 0.01 ml, what total volume should
be withdrawn so that the volume will be known to a precision of 3 parts per
thousand? (Remember that two readings of the buret must be made for every
volume of liquid withdrawn.)
23. At some time or another, nearly everyone must decide whether to reject a
suspicious-looking result or to include it in the average of all the other results.
There is no agreement on what criteria should be used but, lacking information
about errors made in the experimental procedure, a rejection may be made on
the following statistical basis. If d and r are, respectively, the differences
Problems B 63

between the questionable result and the values closest to it and farthest from
it, then there is a 90% probability that the questionable result is grossly in
error and should be rejected if
for 5 values, dlr > 0.64
for 4 values, dlr > 0.76
for 3 values, dlr > 0.94
Which, if any, result should be rejected from the following series of measure-
ments?
(a) 9.35, 9.30, 9.48, 9.40, 9.28 (c) 2534, 2429, 2486
(b) 9.35, 9.30, 9.48, 9.32, 9.28 (d) 2534, 2429, 2520