Preparing for a Data Integrity (DI)

Audit

Miguel Pagan
Compliance Consultant
miguel.pagan@agilent.com

September 2017
Agenda

• Data Integrity / Data Life Cycle

• Quality Culture.
• Good Documentation Practice (GDP - ALCOA).
• New approach to audit.
• Data Integrity - Risk Assessment.
• Data Integrity - Audit Preparation.
• Data Review (Internal / External).
• Anti-Fraud auditing.
Data Integrity / Data Life Cycle?
Data Integrity / Data Life Cycle?
Data Integrity

The extent to which all data is complete, consistent and

accurate throughout the data life cycle.

Complete Consistent Accurate

Data Life Cycle

The data life cycle covers data generation, processing,

reporting, archival, retrieval and destruction.

Generation Processing Archival Retrieval Destruction

Source : Data Integrity Definition Guidance (Mar 2015)

Quality Culture

• Data integrity issues

occur and are identified Time
by auditors as a direct Constraints
result of poor quality
culture within
organizations. Unprofessional Resource
Attitude Constraints
Poor
Quality
Culture

Inadequate Inadequate
Training Procedures
Good Documentation Practice
(GDP – ALCOA)
Good Documentation Practice
(GDP – ALCOA)
WHO performed the
analysis?
Attributable

DOES the record

accurately reflect
the events that CAN the data be
Accurate Legible
took place? read and
understood?
ALCOA

IS it the original record? Contempo WHEN and WHERE

Original
IS it the electronic raneous was the data created /
record? recorded?
IS it Meta data?
New Approach to Audit
Change in Regulatory Focus: FDA HPLC
2005 to 2010 Cause 2011 to 2015
% "Cause" of FDA HPLC Warning % "Cause" of FDA HPLC Warning
Letters (2005-2010)
25 Letters (2011-2015)
25 70
21 21 60
60
20
50
15 40
11
10 30
7 7
20 15
5
4 4 8
6 4
10 2 2 2
0 0 0 0 0 0
0 0

Range of Reasons Fewer “Technical” Reasons

Calibration / Qualification Largest Data Integrity Largest
New approach to Audit

• Focus - Potential for fraudulent activity

within your quality systems.
• Assumptions:
• Will assume fraudulent activity is taking
place if they identify weaknesses in your
quality systems.

• “Guilty until proven innocent” approach

to auditing!

• “Data too good to be true!”.

New approach to Audit

• Electronic data (Meta data) is - preferred choice for

regulatory authorities as this is the original (“official”) data.
• Meta data = data about data.
• Meta data is dynamic and can be queried / searched /
trended.
• There is a much higher probability of identifying fraudulent
activity within an organization if Meta data is reviewed.
• If you state that paper is your original raw data in your
internal procedures this will alert an auditor that you are
probably not managing and reviewing electronic (meta)
data.
New approach to audit - Flat data vs. Meta Data

Flat
Printed Pdf Photo
Data OR OR
Chrom Chrom Chrom
file
• Analyst can reprocess data many time and chooses when to print, pdf or copy
the final chromatogram / result from CDS.
• DOES NOT PROVIDE FULL TRACEABILITY AS NO SUPPORTING DATA!

Acq + Proc + + Audit Data

Chrom
method method trail file
• Provides full traceability as supporting data provides evidence how final
chromatogram / result has been generated!
New approach to Audit

• 5 key Data Integrity (DI) questions:

• Is electronic data available?
• Is electronic data reviewed?
• Is meta data (audit trails) reviewed regularly?
• Are there clear roles and responsibilities?
• Has the system been validated for its intended use?

• The answers to the above questions will determine whether companies

are in compliance with 21 CFR part 11 (Electronic records and
signatures).

• Leave the Original Meta data in the CDS and review / approve
electronically to avoid increased Data Integrity risk (the paperless lab).
Data Integrity – Audit Preparation
Data Integrity – Audit Preparation

• Audit Strategy:
• Starts with a specific result (or record).

• Re-create the sequence of events that occurred at the time the result
(or record) was generated using the electronic (meta) data.

• The auditor will want to know:

• WHO performed the analysis?
• WHAT equipment was used to perform the analysis?
• WHEN the analysis was performed?
• WHY the analysis was performed?
• WHERE the electronic (meta) data is stored?

• Answers to the above may lead to more detailed questioning /

inspection.
Data Integrity – Procedures / SOPs
Data Integrity – Procedures / SOPs
• The auditor will expect a suite of SOPs to be in place to support Data
Integrity and minimize risk within your company.

• Examples of typical SOPs include:

• IT policies.
• System administration (CDS access, roles and privileges).
• Data management and storage.
• Data acquisition and processing.
• Data review and approval.
• Date archiving and back-up.
• Anti-fraud monitoring.
IT Infrastructure
IT Infrastructure

• Server room:

• The room is secure.

• IT access only.

• Tidy and in good working

order.

• Has back-up and disaster

recovery procedures in place.

• Date/time functionality of
servers are correct.
IT Infrastructure

• The auditor will select a number of

instrument controlling PCs within the lab
and check:
• Date/time functionality is correct.
• Date/time cannot be changed by the
lab personnel.

Confirm that date/time functionality

on all PCs within the lab is locked
down and can only be changed by
IT personnel with Administration
privileges.
Administration
Administration

• The auditor will want to

understand how access to the
Chromatography Data System
(CDS) is authorized and
controlled.

• You will need to justify the

access levels within the CDS
and the user privileges at each
level.
 Administration
• Specific user profiles and passwords required to access instrument
software and provide audit trail traceability.
• Administration control should be independent of Analytical function to
eliminate conflict of interest.
• Clear segregation of duties with no overlap of privileges.

User: dbrown User: cwallis

Profile: Administrator Profile: Super User
Password: ******** Password: ********

User: bthompson
User: asmith
Profile: Data Reviewer
Profile: Analyst
Password: ********
Password: ********
Administration

• Reinforce – DO NOT SHARE PASSWORDS.

• Password policies - changed on a regular
basis to protect your profile.
• Password strength - mix of alpha numeric
characters and have a high strength.
• User policies - need to log-off the CDS
immediately after use to avoid profile
potentially being used by other personnel to
acquire, process or manipulate data.
• User profiles - set to auto-lock after a period of
inactivity to protect the user profile and data
within the CDS.
Administration

• The regulatory auditor will want to confirm

that the Audit Trail functionality is switched
ON within the CDS Admin console.

• The regulatory auditor may ask for

Administration reports:
- Active users
- User privileges
- Administration audit trail report
Administration

• Specific privileges within the

user profile:

- They will want assurance

that data cannot be deleted
by a user once acquired.

- They will want to know if

data can be moved to a
different folder to potentially
“hide” it. (e.g. trial injections)
Administration

- They will want to see

that electronic data that
has been processed
must be saved before it
can be submitted for
review (or printed to hard
copy).

Make sure you

understand the privileges
applied to each user
profile and be prepared to
justify to the regulatory
auditor.
Data Management
Data Management

• The auditor will want to understand how

data is managed within the CDS and
check that users are following the
internal procedure.
• Define a data management structure
that segregates different types of data
and enables easy retrieval during the
audit.
• Segregate GMP release data from

Segregation
Research / Development data if you
have dual functionality within your
organization using the same CDS /
Server.
Data Management

• Data structure - Consider

what types of data you
produce and decide how
each type of data should be
stored within the CDS.
Data Management

• Periodic GMP data archiving – make

sure that data archiving is defined in
your procedure and performed
regularly.
• This approach minimizes the amount
of “live” data that can be accessed by
users and potentially reprocessed to
change previously reported results.
• The users should not have access to
archive folder(s) which adds an
additional layer of protection to the
electronic data.
Data Processing
Data Processing

Calculated
Creation / / Reported Storage /
Processing Destruction
Acquisition Result Archiving

• Data Processing Risks:

• Main area where results can be manipulated by human
intervention.

• Target area for auditors.

• Controlled by procedures, user access and locked methods.

• Avoid multiple reprocessing (if possible)!

Data Processing

• All data processing should be

performed within the CDS for
system suitability and batch
results wherever possible.

• Move away from using validated

excel spreadsheets (no longer
meta data).

• For commercial release testing

the auditor will expect
processing methods to be
validated and locked by the
administrator.
Data Processing

• Use pre-defined integration

parameters wherever possible
to avoid manual integration of
multiple peaks.

• Chromatography should be
presented on an appropriate
scale so that integration is
clearly visible.

• Disable annotation tools within

the CDS (electronic whiteout!)
which could be used to
deliberately alter the
appearance of the
chromatograms.
Data Processing

• Save all changes to individual

chromatograms, sequences
and processing methods before
submitting for review.

• Ensure that accurate audit trail

comments are entered into the
CDS when prompted to provide
traceability.
Audit Trail Comment Audit Trail Comment
Integration parameters
dhsjdhsjjsjdksd
updated
Internal Data Review
Internal Data Review
• Parameters to check:

• Analysis performed as per the monograph.

• Sequence information correct.

• Chromatography is typical.

• SST acceptance criteria achieved.

• NO “conditioning” or “test” injections using the sample (use a

standard or control sample if specified by your procedures and
monograph).

• Correct integration (pay attention to MANUAL integration).

• Chromatography appropriately scaled.

Data Review

• Individual results meet specification.

• Check the sequence and individual injection audit trail - any atypical /
suspect activity?

• Data processing:
- Do the audit trail comments provide traceability?
- Can the reprocessing be justified?

• Check electronic results within the CDS match results reported on hard
copy chromatography or in LIMS / SAP systems.
External (Auditor) Data Review
External (Auditor) Data Review

• Auditor checklist:
• Administration control.

• Individual user profiles and passwords.

• Clear segregation of duties within user profiles.

• Restricted privileges for user (can’t delete / over-write / move).

• Audit trail functionality switched ON.

• Date / time functionality locked by IT.

• Provide Demonstration – User log-on (multiple), date / time

locked, can’t delete data.
External (Auditor) Data Review

• Auditor checklist:
• Data recall – Electronic sequence / data file recall in lab using staff
member. Data recall needs to be fast and efficient.

• Data review – Chromatography scaling, integration and electronic

results.

• Audit trail review – looking for suspicious activity, justification of

processing.

• Training – assess staff competency with CDS in lab. Make sure

staff are trained to interact with the auditor. Have a CDS super-
user present during the lab inspection.

• Query search –assurance that sample hasn’t been analyzed

multiple times as part of an investigation.
External (Auditor) Data Review

• Auditor checklist:
• Final electronic results in CDS match those reported on C of A.

FDA / MHRA inspectors have been

trained by Data Integrity and CDS
experts!

They have detailed knowledge of your

CDS and know where to find the meta
data to identify if fraudulent activity has
taken place!
Anti-Fraud Monitoring
Anti-Fraud Monitoring

• Expectation:
• Anti-Fraud policies / procedures to be available.
• Regular internal anti-fraud audits looking at different areas within your
company / department.
• Documented evidence of anti-fraud audits with associated CAPAs for
audit findings.
• QA training for CDS to perform audit trail review before GMP batch
release.

• Consider:
• Having a Data Integrity / Anti-Fraud officer.
• Perform spot-checks on lab operations outside the regular audit
schedule.