Empower Data Review and Approval

Review and Approve Results in Empower
Data, Meta Data and Audit Trails
©2013 Waters Corporation 1

What is an audit trail?
 Systematic “story” of the data from creation, through

interpretation and final assessment and reporting
 Easily confused as : Log of everything that happens in my
system”
– Recent quote: “We switched on audit trails of everything at the
beginning” and.......

Process Understanding
 Key to determine the true audit trail

– How is data recorded, where and when
– What additional records are created along the way
– How are these connected and what traceability is there between
them
– If records are not editable, do you need a traditional audit trail
functionality?
– Can you create a paper audit trail / change control?
 Using the answers to these questions the criticality of the audit
trail can be determined and a sensible review process designed.

Critical Audit trails
Instrument fit
Standard for us Training
Solution and Methods for Processed
(Maintenance records of Raw Data Reported data
Sample interpretation results
and calibration operators
Preparation
records)
 At each step, where are records created?

 Can they be modified? By whom? And how is this traced
 How is data passed from one system to another?
– Automatically and validated
– Manually and Four eyes checked?

Annex 11: Prompting the question of
audit trail review

Four new key areas in Annex 11
 Supplier Audits: including the requirement to share a summary

of your assessment
– Be sure this is agreed in your vendor NDA agreement
 Qualification of IT Infrastructure
– And a formal agreement with IT departments
 Inclusion of Risk Management
– In Regulation rather than in Guidance
 Review of Audit Trails
– Specifically mentioned

Accurate and Traceable Data Entry
- Audit Trails
 Annex 11§9
– Based on Risk
o Record of all GMP relevant changes and deletions
o Reasons should be included
o Convertible to human readable form and regularly reviewed.
 Additionally 11§8.2
– ..ability to “ generate printouts indicating if any data changed since
original entry”

Review of Audit Trails
 Audit trails need to be available and convertible to a generally

intelligible form and regularly reviewed. (A11§9)
– Part 11 “ agency review”
 From a NIST publication*
– Audit trails are a technical mechanism that help managers
maintain individual accountability. ..Users are less likely to
attempt to circumvent security policy if they know that their actions
will be recorded in an audit log.
– “Determine how much review of audit trail records is necessary”
 Increased appearance of Warning Letter observations
* Introduction to Computer Security: The NIST Handbook

US FDA Audit Trail Review Warning
Letters
 Gulf Pharmaceutical Industries Feb 2012

– We also note that your SOP does not have provisions for any
audit trail reviews to ensure that deletions and/or
modifications do not occur.
 Banner Pharmacaps Sept 2006:
– A second person must review these audit trails, particularly given the
lack of controls for preventing data manipulation. Such an audit
may well have detected the data manipulation which was
occurring at your facility.
 Sunrise Jan 2010:
– In addition, your firm's review of laboratory data does not
include a review of an audit trail or revision history to determine
if unapproved changes have been made.

Recent Cases of Deliberate Fraud
Using e-Records…

Empower Compliance Features:
Audit Trails

Biggest Compliance feature in
Empower??
 The built in Empower Database

– Enables every object to be uniquely referenced
– Can never overwrite data
– Can never mistake which data went with which method
– Ensures easy and accurate data review
 Automatic versioning for results / methods

– With full computer generated audit trail
– WHO changed WHAT (before and after values) WHEN.... And WHY?)

Empower Audit Trails ID numbers
 Empower is built into an Oracle Database

 This database gives each object or result a Unique Identifier for
tracking the values and records
 This identifier is unique within each project.
 Modification of any data base object results in a NEW record
with NEW identifiers
 Many users of Empower use these ID number to prove and

identify results to auditors
– Also to track for their own purposes

Assigns Unique ID # for All Entries
 Raw Data ID’s ID’s

 Result
– Missing numbers from
 Missing numbers
Channel assigned to
Calibration
o Assigned to Injection
o And....1142,
Assigned to Vial
1143, 1144,
1145, 1146?

IDs assigned to Cal Curve in Peak
View

Method ID, Version and ’Locked’

Result Audit Trail…
more than a table of changes
Sample Standards used

Sets for Calibration
Calibration
Original Curves
Processing Method
Unchanged
Raw Data
File
Unique
Result
Original
Instrument Method
Who Collected When

Who Processed What
Who Reviewed Why
Product Code/
E-cord information Who Approved
Stage Reagent
©2013 Waters Corporation LC/GC System Used LIMS ID 17
Built in Audit Trails in Empower
 It is not possible to create, manipulate, modify or delete data

inside Empower without creating an audit trail entry
 All user actions are logged in various audit trails and associated
with the logged in USERNAME
– Assumes all users have unique User Account
 Multiple “modes” of audit trail
– Silent
– Full – Includes the requirement to enter a reason “Why?”
o With free form reasons
o With predefined reasons only (Default Strings)
– Reauthentication (re entry of password to confirm identity)
 Empower Audit trails are not editable or modifiable by ANY
USER

Empower Audit Trails
 Sample Audit Trail

– Tracks changes to entered data about each sample
 Result Audit Trail
– Links results to instruments, samplesets, methods, calibration
curves and standards used in calibration.
– Also traces any manual manipulation of data
 Method Audit Trail
– Keeps all versions of method for recreation of results
– Audit Trail monitors each change, before and after values, who
when and why
– Different versions can be compared to identify the differences

Empower Audit Trails
 Project Audit Trail

– Gives overview of all changes in a project
– Includes details of method / data deletion
 System Audit Trail
– shows changes to system objects and system policies
– details archive activity
– notes all changes to security (users, user types etc)
– documents all successful and unsuccessful logins
o you have a history of who was logged into the application at any
time
o you have information about system break in attempts
o includes the client the login/login attempt occurred at

Empower Sample Audit Trail

Method Properties and Audit Trail

Method Differences

How to Review Audit Trails

Reviewing Audit Trails
 Purpose
– To uncover possible cases of fraudulent behaviour
o Multiple processing data
o Altering metadata to make results pass
o Hiding or altering meta data on reports sent to QA
o Uncovering persistent suspicious behaviour around security of data
o Ensuring only authorised users have access to certain functionality
o Deletion of data
o Altering system policies /configuration / settings without change
control procedures
 How?
– Two possible procedures:
o Review of relevant audit trails as part of data review
o Periodic review of system level audit trails by admin

Review of Audit trails
 Review audit trails as part of data review process

– Find anomalies before batch release
– Focus of user behaviour that affect results
– Peer Review / Manager review / QA review?
 Periodic Review of overall Audit trails
– Looking for system level activity without correct documentation,
change control, testing or approval
o Eg changing system policies, user access or deletion of data
 Biggest issue: Audit trails are often more a log of all activity (to
comply) and not designed for easy review
– But it is expected that inspectors will look at the audit trails

How to Review Audit Trails in
Empower?
 Use the Preview and Sign off tools

– Inform regulatory bodies that electronic signatures are being used in
your company
– Set up system policies to control signature processes
– Create reports including all relevant meta data and include hte Sign
Off property
o Sample Set, Method details, All versions of Results and all Audit
trails
– Review all pages of report
– Sign report

Using Reports and E-Signatures
to review and approve results

Selecting policies for GXP and eSigs
 Result Sign off tab

– Sign off Inactivity delay
– Clear password
– Enforce single logon
– Allow lock channels
– Save report image as pdf
– Multiple sign off behavior
– Enforce sign off 1 before
sign off 2

Report methods for electronic
Signatures
 Report methods to be used for electronic signatures must have

‘Allow this method to be used in sign off’ checked
 If the content of the method is
changed this check box will be
automatically deselected and
must be selected again before
the method can be used for
electronic signatures

Report methods for electronic
Signatures
 Viewing sign off information in the report

 Add the sign off table to the method

Signing reports
 Report is viewed as normal

 Signature table is displayed

Signing reports
 Review the report and if suitable

 Click the sign off button
 Note
 Sign off button is greyed
out until all the pages of
the report have been
displayed

Signing reports
 Sign off dialogue box is displayed

 Enter
– User name
– Password
– Reason
 Click sign off
 If signing of for ‘Sign off 2’
– Can lock channels

Signing reports
 Report now contains sign off information

What should be included in an
Electronic Signature report

 Should I use a report to review data

 What should go on that report
– As much as possible
– Only what is essential

 Individual Reports or Summary?

Individual or Summary Reports
 Individual: Each result is signed individual date/ time

– Can reject one result out of a batch
 Summary: One signature applies to each individual result
– Same date and time
– Simplify the signature box on reports with single signing action
 What to do with results which are not signed?

– Are they superseded results?
– Do you need to sign as “rejected”?
– What will you say to an auditor who asks about these results?

 Chromatogram
– One... Some... All... None
– Full
– Expanded base line
– Thumb nail
 How useful is this for

scientific review?
– No zoom on integration
 Could annotations have
obscured data?
– Only show up on printed
reports, not electronic
reports

 Method data
– Processing... Instrument... Method Set... Sample Set... None
– Some... All...
– History
– ID
– Version
 Compare Methods
 Train reviewer on
Empower traceability

 Sample Set Method?

 Sample Set?
 Channel Information (is it complete? Test injections?)
– Samples... Standards... Controls... Blanks... None
– Some... All...
– Sequence, ID, Channel name

 Result Information
– Some... All...
– ID
– Result Set Name
– Result Set ID
– Altered
– Manual
– Peak Codes

 Calibration information
– One... Some... All... None
– Curve plot
– Equation
– Calculations
– ID

 Audit trails
– Some... All...
– Sample History
– Method History
– Method Versions

Preview and Reports?
 Advantage: Evidence of “review”

 Issues:
– Just because they are on the reports and you looked at the at page,
did you actually see anything irregular?
– Can you pick out “ normal” audit records from “ non normal”?
– What real value does this bring to the review process?
– Creating huge reports for each run, effectively a new “ copy” of the
electronic data.

Empower?
 Review chromatograms, methods and relevant Audit Trails in

Empower application
– Sign a report to document that you have followed the review SOP
I sign this data to attest that I performed/ reviewed / approved

this data according to SOP 12345
This includes review of all manual entry of meta data (eg
weights, dilution and other factors and also the relevant audit
trails.)
– Mimics the review of many other aspects of a method execution

o Sampling / solution creation/ standard creation
• No true recording of activity

How to review data in Empower
 Use the Empower project name to open the correct Empower

project
 In the result or result set tab search for the ID

 Use right click “View as...” to select

– Sample Sets
– Results
– Peaks
– Processing methods
– Instrument Methods
– Sample Set methods
– Audit Records

 In Results tab
– Result ID, Processed Manually, number of results for channel, result
number, processed by, processed date, Processing method, date acquired,
channel, vial, injection, system, label, sampleset id, run time, custom
fields, faults, blank, comments, peak codes, number of sign offs, sample
name.......

 In Sample Set use alter sample

 See if a sample has been altered then right click and
Select view
– Sample History
– SampleSet method
History

 Sample History
– Who
– What
– When
– Why

 SampleSet/Instrument/Processing Method History

– Name
– Date
– ID
– Version
– Audit Trail
– Check differences between
version

Empower?
 Concerns
– Difficult to FIND all audit records in Empower
– Complicated SOP to follow
o Even for experienced Empower users
o Difficult for non/ new/ infrequent users like QA
– No automatic documentation of the review process

The Review Tool
 Access to integrated chromatograms /results

– All integration positions
– Ability to zoom in to examine without reprocessing
 Peak and Result level values
 Instrument Method
 Processing Method
 Calibration Curves Sample Standards used
for Calibration
Sets
 NOW Direct access Sample Set
 All that was missing was Original
Calibration
Curves
the audit trails
Processing Method
Unchanged
Raw Data
File
Unique
Original Result
Instrument Method
Who Collected
Who Processed When
Who Reviewed What
Product Code/ Who Approved
E-cord Stage Reagent
Why
information LC/GC System Used
LIMS ID
Reviewing Audit Trails:
A New tool in FR2
 Designed to make the requirement to

review Audit trails simpler
– Annex 11 and various warning letters
 Launched from Review
– Where a manager would be looking at Results
 Brings into one window audit records from
– Project window
– Manual results
– Method changes
o Processing, Instrument, Sample Set (alter sample) and Method Set
o Allows multiple methods to be compares
 Compares results from superceded results
– Where results have been reprocessed
– Compares Areas, RT, Amount etc between two results

Enhanced Data Review
New Result Audit Viewer(RAV)

Result History Table
 Searches the multiple Audit trails for data related to the

selected injection
– Acquisition Log
– Injection Log
– Sample Set History (Includes sample set level history, sample level
history and method changes)
– Sample History
– Method History
– Audit Trail Records
 Merges data, with different columns and information, into one
easy to search table..
– Some logs are missing ACTION REASON or type... So a N/A is
entered
– Blank columns really were not populated

Enhanced Data Review
New Result Audit Viewer
Result History

Result Differences
 Middle table show all superseded results that are stored in the
CURRENT result set
– Individual results can be selected and brought into review to see “
lone results or across result sets
 Selecting these results will compare to the most recent result
– Selected in the top table
 Select a peak to see peak values and compare across results
– Faults are still highlighted
– Differences can be highlighted
– Can select to ONLY show differences
o All can be saved as preferences, along with column selection and
order

Result Differences

Result Audit Viewer
Differences in
the results are
in blue.
Results outside
limits are in
red
Manual
integration
noted in
Integration
Type field

Method Information and Differences
 Selecting a result in the TOP table will search and display the
associated methods in the middle table
 For each type of method an entire method history can be
displayed in the middle table
– Although easier to see in Review graphically
 The bottom table displays the contents of the method version
selected
 Selecting two method versions allows methods to be compared
– Uses the regular “Method Differences” functionality
o Show Only or Highlight differences
– Can only compare two methods at one time
– NB for details of change, use the Result History tab

Processing Method Comparison

Result Audit Viewer
Methods
differences
shown in
separate tab
Differences
have text
search

Questions How would you use this?
 As part of Chromatogram review for all chromatograms?

 Only to investigate where odd integration is seen,
– Where Manual integration is done?
– Where data faults or is close to a fault?
 On a random number of samples?
 Would you update SOPs to use this tool?

– Would you test them
– And that they were being followed?
 Would you want more information in here?

 How would you document that Audit trail review was done?
 Could we add more tools?

Empower Data Review and Approval

Uploaded by

Copyright:

Available Formats

Empower Data Review and Approval

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Empower Data Review and Approval

Uploaded by

Copyright:

Available Formats

Review and Approve Results in Empower

Data, Meta Data and Audit Trails

©2013 Waters Corporation 1

 Systematic “story” of the data from creation, through

©2013 Waters Corporation 2

 Key to determine the true audit trail

©2013 Waters Corporation 3

 At each step, where are records created?

©2013 Waters Corporation 4

©2013 Waters Corporation 5

 Supplier Audits: including the requirement to share a summary

©2013 Waters Corporation 6

©2013 Waters Corporation 7

 Audit trails need to be available and convertible to a generally

* Introduction to Computer Security: The NIST Handbook

©2013 Waters Corporation 8

 Gulf Pharmaceutical Industries Feb 2012

©2013 Waters Corporation 9

©2013 Waters Corporation 10

©2013 Waters Corporation 11

 The built in Empower Database

 Automatic versioning for results / methods

©2013 Waters Corporation 12

 Empower is built into an Oracle Database

 Many users of Empower use these ID number to prove and

©2013 Waters Corporation 13

 Raw Data ID’s ID’s

©2013 Waters Corporation 14

©2013 Waters Corporation 15

©2013 Waters Corporation 16

Sample Standards used

Who Collected When

 It is not possible to create, manipulate, modify or delete data

©2013 Waters Corporation 18

 Sample Audit Trail

©2013 Waters Corporation 19

 Project Audit Trail

©2013 Waters Corporation 20

©2013 Waters Corporation 21

©2013 Waters Corporation 22

©2013 Waters Corporation 23

©2013 Waters Corporation 24

©2013 Waters Corporation 25

 Review audit trails as part of data review process

©2013 Waters Corporation 26

 Use the Preview and Sign off tools

©2013 Waters Corporation 27

©2013 Waters Corporation 28

 Result Sign off tab

©2013 Waters Corporation 29

 Report methods to be used for electronic signatures must have

©2013 Waters Corporation 30

 Viewing sign off information in the report

©2013 Waters Corporation 31

 Report is viewed as normal

©2013 Waters Corporation 32

 Review the report and if suitable

©2013 Waters Corporation 33

 Sign off dialogue box is displayed