Applying Trend Analysis Techniques to Quality Control Measurements in Radionuclide Metrology

Simon Jerome, NPL

LSUF, NPL, 24th January 2006

LSUF 24th January 2006

Quality Control ISO 17025:2005 5.9 Assuring the quality of test and calibration results 5.9.1 The laboratory shall have quality control procedures for monitoring the validity of tests and calibrations undertaken. The resulting data shall be recorded in such a way that trends are detectable and, where practicable, statistical techniques shall be applied to the reviewing of the results. This monitoring shall be planned and reviewed and may include, but not be limited to, the following: a) regular use of certified reference materials and/or internal quality control using secondary reference materials NOTE The selected methods should be appropriate for the type and volume of the work undertaken.

Event and date

LSUF 24th January 2006

Two Techniques Commonly Used (More Are Available) Shewhart more easily interpreted CuSum (Cumulative Summation) more sensitive to change

Event and date

LSUF 24th January 2006

Use of Shewhart charts Common usage is to set 2 and 3 limits, based on 11-31 replicate results Action taken if 3 limit is exceeded expected 3 times in 1000 Warning is registered if 2 but not 3 limit is exceeded expected 5 times in 100 Straightforward, but no warning signals before warning or action levels breached

Event and date

LSUF 24th January 2006

Shewhart Control Chart - RMG Ionisation Chambers Radium Check Source Measurement

50.3650

50.3600

Ion Chamber Current/pA

3 exceeded action required immediately, since the procedure is out of control. Measurement should stop until the cause of the action limit violation is identified and corrected.

50.3550

50.3500

50.3450

50.3400

50.3350

50.3300

50.3250

50.3200

50.3150 1-Aug

3-Aug

5-Aug

7-Aug

9-Aug

11-Aug

13-Aug

15-Aug

17-Aug

19-Aug

21-Aug

23-Aug

25-Aug

27-Aug

29-Aug

31-Aug

2-S e p

4-S e p

Event and date

Date

LSUF 24th January 2006

Shewhart Control Chart - RMG Ionisation Chambers Radium Check Source Measurement

50 .3 65 0

50 .3 60 0

2 exceeded situation needs to be closely monitored

Ion Chamber Current/pA

50 .3 55 0

50 .3 50 0

50 .3 45 0

50 .3 40 0

50 .3 35 0

50 .3 30 0

50 .3 25 0

50 .3 20 0

50 .3 15 0 1-Au g 3 -Au g 5 -Aug 7-Aug

9-Au g 11-Au g 13 -Au g 15 -Aug 17 -Aug 19-Au g 2 1-Au g

23 Au g

25 Au g

2 7Aug

2 9Aug

3 1-Au g 2-S e p 4-S e p

Event and date

Date

LSUF 24th January 2006

ISO 8258:1991 Shewhart Control Charts Introduces the concept of Trend Analysis Relies on the range observed in measurement: rj = |x(j+1) xj| X=

x
j

' j

N'
R=

r
j

' j

( N 1) '
Event and date 7

LSUF 24th January 2006

ISO 8258:1991 Shewhart Control Charts Action limit set at: Warning limit set at: Information limit set at: X R2.66 X R1.77 X R0.89

Based on a series of single observations; limits vary depending on the batch size and subgroup

Event and date

LSUF 24th January 2006

Shewhart Control Chart - RMG Ionisation Chambers Radium Check Source Measurement

50 .3 6 50

50 .3 6 00

Warning Limit Exceeded twice in under 42 measurements. Measurement should stop until the cause of the action limit violations are identified and corrected. (ISO 8259:1991 suggests 2 out of 3 is an action point)

50 .3 5 50

50 .3 5 00

Ion Chamber Current/pA

50 .3 4 50

50 .3 4 00

50 .3 3 50

50 .3 3 00

50 .3 2 50

50 .3 2 00

5 0.3150 1-Aug

3 -Aug 5-Aug

7-Aug 9-Au g 11-Au g 13-Au g 15-Au g 17 -Au g 19 -Au g 21-Au g

23 Au g

25 Au g

27Au g

29Aug

3 1-Aug 2 -S e p 4 -S e p

Event and date

Date

LSUF 24th January 2006

Shewhart Control Chart - RMG Ionisation Chambers Radium Check Source Measurement

5 0 .3 6 5 0

5 0 .3 6 0 0

Nine points in a row on one side of the mean. This may indicate that there has been a shift in the mean; work can continue, however the cause of the shift needs to be identified and the limits re-set if justified

5 0 .3 5 5 0

5 0 .3 5 0 0

Ion Chamber Current/pA

5 0 .3 4 5 0

5 0 .3 4 0 0

5 0 .3 3 5 0

5 0 .3 3 0 0

5 0 .3 2 5 0

5 0 .3 2 0 0

5 0 .3 15 0 1-Au g

3 -Au g

5 -Au g 7 -Au g 9 -Au g 11-Au g 13 -Au g 15 -Au g 17 -Au g 19 -Au g 2 1-Au g

23Au g

25Au g

27Au g

29Au g

3 1-Au g 2 -S e p 4 -S e p

Event and date

Date

10

LSUF 24th January 2006

Shewhart Control Chart - RMG Ionisation Chambers Radium Check Source Measurement

5 0 .3 6 5 0

5 0 .3 6 0 0

Six points in a row steadily increasing or decreasing. Suggests that the measurements are shifting with time; needs investigation, since the action and warning limits will eventually be exceeded

5 0 .3 5 5 0

5 0 .3 5 0 0

Ion Chamber Current/pA

5 0 .3 4 5 0

5 0 .3 4 0 0

5 0 .3 3 5 0

5 0 .3 3 0 0

5 0 .3 2 5 0

5 0 .3 2 0 0

5 0 .3 15 0 1-Au g

3 -Au g

5 -Au g 7 -Au g 9 -Au g 11-Au g 13 -Au g 15 -Au g 17 -Au g 19 -Au g 2 1-Au g

23Au g

25Au g

27Au g

29Au g

3 1-Au g 2 -S e p 4 -S e p

Event and date

Date

11

LSUF 24th January 2006

Shewhart Control Chart - RMG Ionisation Chambers Radium Check Source Measurement

5 0 .3 6 5 0

5 0 .3 6 0 0

Ion Chamber Current/pA

Fourteen points in a row alternating up and down. Suggests a cyclical change in conditions; may be due to two different operators

5 0 .3 5 5 0

5 0 .3 5 0 0

5 0 .3 4 5 0

5 0 .3 4 0 0

5 0 .3 3 5 0

5 0 .3 3 0 0

5 0 .3 2 5 0

5 0 .3 2 0 0

5 0 .3 15 0 1-Au g

3 -Au g 5 -Au g

7 -Au g 9 -Au g 11-Au g 13 -Au g 15 -Au g 17 -Au g 19 -Au g 2 1-Au g

23Au g

25Au g

27Au g

29Au g

3 1-Au g 2 -S e p 4 -S e p

Event and date

Date

12

LSUF 24th January 2006

Shewhart Control Chart - RMG Ionisation Chambers Radium Check Source Measurement

5 0 .3 6 5 0

5 0 .3 6 0 0

Ion Chamber Current/pA

Four out of five points in a row exceeding the information level. This may indicate a mean shift

5 0 .3 5 5 0

5 0 .3 5 0 0

5 0 .3 4 5 0

5 0 .3 4 0 0

5 0 .3 3 5 0

5 0 .3 3 0 0

5 0 .3 2 5 0

5 0 .3 2 0 0

5 0 .3 15 0 1-Au g

3 -Au g

5 -Au g 7 -Au g 9 -Au g 11-Au g 13 -Au g 15 -Au g 17 -Au g 19 -Au g 2 1-Au g

23Au g

25Au g

27Au g

29Au g

3 1-Au g 2 -S e p 4 -S e p

Event and date

Date

13

LSUF 24th January 2006

Shewhart Control Chart - RMG Ionisation Chambers Radium Check Source Measurement

5 0 .3 6 5 0

5 0 .3 6 0 0

Ion Chamber Current/pA

Fifteen points in a row between the lower and upper information levels. This may arise after the procedure has been operated for some time and may require the limits to be re-set

5 0 .3 5 5 0

5 0 .3 5 0 0

5 0 .3 4 5 0

5 0 .3 4 0 0

5 0 .3 3 5 0

5 0 .3 3 0 0

5 0 .3 2 5 0

5 0 .3 2 0 0

5 0 .3 15 0 1-Au g

3 -Au g

5 -Au g 7 -Au g 9 -Au g 11-Au g 13 -Au g 15 -Au g 17 -Au g 19 -Au g 2 1-Au g

23Au g

25Au g

27Au g

29Au g

3 1-Au g 2 -S e p 4 -S e p

Event and date

Date

14

LSUF 24th January 2006

Shewhart Control Chart - RMG Ionisation Chambers Radium Check Source Measurement

5 0 .3 6 5 0

5 0 .3 6 0 0

Ion Chamber Current/pA

Eight points in a row either side of the mean with none between the upper and lower information levels. This may indicate a mean shift.

5 0 .3 5 5 0

5 0 .3 5 0 0

5 0 .3 4 5 0

5 0 .3 4 0 0

5 0 .3 3 5 0

5 0 .3 3 0 0

5 0 .3 2 5 0

5 0 .3 2 0 0

5 0 .3 15 0 1-Au g

3 -Au g

5 -Au g 7 -Au g 9 -Au g 11-Au g 13 -Au g 15 -Au g 17 -Au g 19 -Au g 2 1-Au g

23Au g

25Au g

27Au g

29Au g

3 1-Au g 2 -S e p 4 -S e p

Event and date

Date

15

LSUF 24th January 2006

In practice. Use software to record and present the data Require operators to manually carry out a check (use a paper-based check list) for the above conditions Carry out investigations as necessary and institute improvement actions However, it is incorrect simply to: Repeat a measurement and find the chart to be in control Reset the chart parameters without at least seeking a reason Rely on the software to do it for you Rely on manufacturers software
16

Event and date

LSUF 24th January 2006

In addition: Use a CuSum Chart: c1 = x1 X c2 = c1 + x2 X c3 = c2 + x3 X . . . ci = ci-1 + xi X Deviation from control become clear very rapidly
Event and date 17

Tritium counting efficiency (%)

64.3 64.5 64.7 64.9 65.1 65.3 65.5 65.7 65.9 66.1

30 -J un -0 3 19 -A ug -0 3

LSUF 24th January 2006

Event and date

08 -O ct -0 3 27 -N ov -0 3 16 -J a n04 06 -M ar -0 4 25 -A pr -0 4 14 -J un -0 4 03 -A ug -0 4

Shewhart Value Control Chart for TriCarb

Date
22 -S ep -0 4 11 -N ov -0 4 31 -D ec 04 19 -F eb -0 5 10 -A pr -0 5 30 -M ay -0 5

18

19 -J u

l-0 5

LSUF 24th January 2006

CuSum Tritium efficiency

-25.0000 -20.0000 -15.0000 -10.0000 -5.0000

Event and date

Date

30-Jun-03

0.0000

5.0000

19-Aug-03

08-Oct-03

27-Nov-03

16-Jan-04

06-Mar-04

CuSum for Tritium efficiency

25-Apr-04

14-Jun-04

03-Aug-04

22-Sep-04

11-Nov-04

31-Dec-04

19-Feb-05

10-Apr-05

30-May-05

19
19-Jul-05

LSUF 24th January 2006

Tritium Check Source for LSC Shewhart Chart gives first warning on 6th November 9 points in a row, one side of the mean (and many thereafter) CuSum Chart gives first warning around 1st November problem is clear from this point on On-board Q/C software gave first fault message on 30th January 3 limit violation

Event and date

20

Tritium background (counts/second)

30 -J u
10.100 10.600 11.100 11.600 12.100 12.600 13.100 13.600 8.600 9.100 9.600

n03

LSUF 24th January 2006

Event and date

19 -A ug -0 3 08 -O ct -0 3 27 -N ov -0 3 16 -J an -0 4 06 -M ar -0 4 25 -A pr -0 4 14 -J u n04 03 -A ug -0 4

Shewhart Value Control Chart for TriCarb

Date
22 -S ep -0 4 11 -N ov -0 4 31 -D ec 04 19 -F eb -0 5 10 -A pr -0 5 30 -M ay -0 5

21

19 -J ul

-0 5

LSUF 24th January 2006

CuSum Tritium Background

-10.0000 10.0000 20.0000 30.0000 40.0000 50.0000 60.0000

Event and date

Date

30-Jun-03

0.0000

19-Aug-03

08-Oct-03

27-Nov-03

16-Jan-04

CuSum for Tritium Background

06-Mar-04

25-Apr-04

14-Jun-04

03-Aug-04

22-Sep-04

11-Nov-04

31-Dec-04

19-Feb-05

10-Apr-05

30-May-05

22
19-Jul-05

LSUF 24th January 2006

Tritium Background for LSC Shewhart Chart gives first warning on 15th December action limit violation (event on 8th October assigned to 99Mo/99mTc source in vicinity) CuSum Chart gives first warning around 18th December problem is clear from this point on (event on 8th October does not affect CuSum) On-board Q/C software gave first fault message on 15th February 3 limit violation

Event and date

23

LSUF 24th January 2006

Analysis of data Engineer called out in February Deteriorating Photo-Multiplier tube identified 14C check source did not return comparable data, apart from on action limit violation that did not have an assignable cause Subsequent measurements with other systems indicate general instability of 3H measurements observed at other establishments Further investigations reveal hidden parameters that low energy emitters are very sensitive to (such as DBB) Essential to know the system well before use

Event and date

24

LSUF 24th January 2006

Conclusions Full implementation of ISO 8258:1991 allows operators to far better understand measurement systems Potential faults can be trapped by preventative actions rather than corrective actions Simple to use and interpret Applicable to similar systems -spectrometers, Germanium spectrometers, NaI(Tl) -spectrometers, ionisation chambers &c Customer calibrations and source generation can be carried out with greater confidence

Event and date

25

Questions and Feedback