Verification SCACM

“Verifications and Validations:
How to bring a new test to the

lab aiming at clinical
stewardship and compliance”
Maria E. Navas MD DABMM
Microbiology Laboratory Director, Louis Stokes VA Medical Center,
Cleveland.
VERIFICATION DEFINITION
• CLIA, CLSI– a one-time process completed before a
test or instrument is used for patient testing and
conducted to demonstrate that test system performs
according to specification.
• CAP- One‐time process; conducted to demonstrate
that the test system performs according to
specification on FDA cleared, unmodified tests.
VALIDATION DEFINITION
• CLIA, CLSI– Ongoing process after an instrument or
test system has been verified. A validation
demonstrates that the test repeatedly continues to
give the expected results as performed over time and
continues to meet the manufacturer’s claims.
• CAP- One‐time process to establish specifications and
performance for Laboratory developed tests and
modified tests.
VALIDATION
May include:
• Internal and external proficiency testing
• Maintenance and calibration records
• Personnel training and competency assessments
• Parallel testing of duplicate instruments
• Correlation with clinical findings
Validation becomes part of the overall laboratory QA program and

should cover the entire analytical process.
Analytical Verification
VS Clinical Verification
• Analytical: Establishing performance parameters
for a laboratory developed test or a modification
of a cleared test.
• Clinical: Demonstrating that the test system
performs according to specification ( as
established by the manufacturer and approved
by FDA)
CAP Checklist
• GEN 42020 –Verify analytic accuracy and
precision
• GEN 42025 –Verify and document analytic
sensitivity (lower detection limit)
• GEN 42030 –Verify and document analytic
interferences
• GEN 42085 –Verify reportable range
CAP Checklist
• GEN 42160 –Explain significant differences in
results to clients
• GEN 42162 –Verify or establish reference
intervals
• GEN 42163 –Evaluate appropriateness of
reference intervals; take corrective action if
necessary
What is Verified?
Manufacturer’s • Sensitivity,
information for
FDA cleared tests • Specificity,
• Precision,
• Accuracy,
• Reportable Range.
ANALYTICAL ACCURACY
• Agreement between studied test result and

“true” result.
• Comparison study: Comparison of results between

analyzed method and “reference” method.
• Certified reference materials are tested using new
method and results compared to expected values.
Comparison Study
• Test 20 samples that span the entire testing range but do not
exceed it.
• Run samples by both new and comparative method (your
current method or a reference method)
• Calculate average bias and compare to clinically allowable bias
• Compare using linear regression analysis
• Bias plots (difference between results versus value from comparison method)
PRECISION
• Precision is repeatability
• Analyze repeatedly and determine variation.
• Measured usually by running QC samples. CLSI protocol
(EP15-A2) suggests two levels, run 3 times per run for 5 days
(15 replicates in all)
• The Standard Deviation for the measurements is compared to
manufacturer’s reported values (package insert).
REPORTABLE RANGE
• Parameter verified for Quantitative tests.
Includes the range of values that an instrument
or test can detect directly (Analytical
Measurement Range) and the range of values
that can be reported with dilution or
concentration of samples (Clinical Reportable
Range)
Reportable
what?
WHAT WE DO (verification study document)
A. BACKGROUND
B. PRINCIPLE OF THE TEST/SYSTEM
C. VERIFICATION DESIGN AND PARAMETERS
D. SPECIMEN
E. REAGENTS
F. EQUIPMENT
G. PROCEDURE
H. REFERENCE RANGES/ INTERPRETATION CRITERIA
I. RESULT REPORTING
J. VERIFICATION CONCLUSIONS
Before anything
is done: plan
EXAMPLE: AST GN 93 card for Vitek 2
• STUDY DESIGN:
Clinical isolates provided by a local Microbiology Laboratory (after
analysis on the Microscan) will be analyzed on the Vitek 2 instrument
using the AST GN93 card. The isolates will include a mix of the most
commonly isolated gram negatives. Identification and susceptibility
results which include the drugs tested on the AST GN 93 card will be
available in the form of Microscan print-outs for each isolate. Accuracy
will be evaluated through result comparison between instruments.
Reproducibility will be evaluated separately using QC material.
Reportable range is not applicable as the comparison is based on the
final interpretation (S,R,I)
EXAMPLE continued
D. SPECIMEN
Twenty fresh gram negative clinical isolates plated on blood agar plates
with their respective Identification and Susceptibility report print outs.
EXAMPLE continued
H. REFERENCE RANGES
The verification study will be considered successful if the following
parameters are fulfilled:
• >90% categorical agreement (CA)
• >90% essential agreement (EA)
• <1.5% very major discrepancy (vmj)
• <3% major discrepancy/error (maj)
• <5% minor discrepancy/ error (min)
EXAMPLE continued
RESULTS REPORTING
No. Sample ID lot # MIC Reference Correlation Tech initials/date
CA/EA/maj/
vmj/min
1
6
Verification study outcomes
• Test is adequate for clinical use. The test was
successfully verified.
• More data and further verification studies are
required.
• Test is inadequate for clinical use until its
performance parameters can be verified. The
laboratory is unable to verify performance
parameters due to failure of the study.
Test
implementation
checklist
LOUIS STOKES VA MEDICAL CENTER
MICROBIOLOGY LABORATORY
MICROBIOLOGY STAFF LOUIS STOKES VA MEDICAL CENTER
COMING BACK TO THE EXAMPLE
•Why did we choose the AST93 card?

• Our Clinical Microbiology Advisory Group
(CMAG) concluded it was the card that fulfilled
our clinical needs.
CMAG???
• Chief of the Infectious Disease Department
• Infectious Disease staff clinician
• Infectious Disease Pharmacist
• Microbiology Laboratory Director
• Microbiology Laboratory Supervisor
• Infection Control Representative (occasional)
What we do…
We meet once a week for one hour and have a recurring agenda:
Issues encountered by clinicians
Issues encountered by Laboratory Staff
Education opportunities
Topics discussed:
• Testing menu/ New testing platforms and kits
• Rejection, acceptance criteria/ Policies
• Extent of specimen work up
• Antimicrobial Susceptibility testing and policies
• Results Reporting
• Specimen collection and submission
Goals
• To analyze the clinical importance and value of the various
components of testing in the microbiology laboratory
• To provide clinical recommendations and guidance for the regular
operations of the laboratory
• To involve clinicians and other costumers in the decision making
process of the laboratory in order to provide targeted and more
efficient services.
• To gain “buy-in” in the different projects of the laboratory not only
from our costumers but also from the Medical Center administration.
So far…
• Antibiograms and problems in the susceptibility testing process have
been identified and solved.
• Creation of current antimicrobial susceptibility testing policies. VERIFICATION
• New, clinically relevant reporting verbiage was proposed by the group
and adopted by the laboratory.
• The critical value policy was reviewed and modified successfully.
• Analysis and justification of new testing platforms. VERIFICATION
• A large scale revision of the testing menu is underway with the goal of
creating a comprehensive, dynamic, efficient and accurate menu.
References
• Robert J. Tibbetts, Verification and Validation of Tests Used in the Clinical Microbiology
Laboratory, Clinical Microbiology Newsletter, Volume 37, Issue 19, 2015, Pages 153-160.
http://dx.doi.org/10.1016/j.clinmicnews.2015.09.004.(http://www.sciencedirect.com/scienc
e/article/pii/S019643991500077X)
• CAP Laboratory General Checklist
• CLSI EP5-A2: Evaluation of Precision Performance of Quantitative Measurement
Methods (2004)
• CLSI EP9-A2: Method Comparison and Bias Estimation Using Patient Samples (2002)
• CLSI EP15-A2: User Verification of Performance for Precision and Trueness (2005)
• CLSI EP17-A: Protocols for Determination of Limits of Detection and Limits of
Quantitation (2004)
• CLSI C28-A2: How to Define and Determine Reference Intervals in the Clinical Laboratory
(2000)

Verification SCACM

Uploaded by

Copyright:

Available Formats

Verification SCACM

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Verification SCACM

Uploaded by

Copyright:

Available Formats

“Verifications and Validations:

How to bring a new test to the

Validation becomes part of the overall laboratory QA program and

• Agreement between studied test result and

• Comparison study: Comparison of results between

•Why did we choose the AST93 card?

You might also like

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.