Managing The Business Risk of Fraud

Managing the Business Risk of Fraud using Sampling and Data Mining
Mike Blakley
Presented to:
Fall 2009
Managing the business risk of fraud using sampling and data mining EZ-R Stats, LLC
PWC Global Survey Nov, 2009

Economic crime in a downturn

Sharp rise in accounting fraud over the past 12 months Accounting fraud had grown to 38 percent of the economic crimes in 2009 Employees face increased pressures to :
meet performance targets keep their jobs keep access to funding
Managing the business risk of fraud
EZ-R Stats, LLC
Survey findings

Greater risk of fraud due to increased incentives or pressures More opportunities to commit fraud, partially due to reductions in internal finance staff While companies are expecting more fraud, they have not done much People who look for fraud are more likely to find it
Managing the business risk of fraud EZ-R Stats, LLC
Session objectives

Understand the framework for managing the business risk of fraud Plan, perform and explain statistical sampling in audits Reduce audit costs using data mining, sequential sampling and other sampling techniques Apply SAS 56, the new SAS suite and the revised (2007) Yellow Book. Run, hands-on, the most productive analytic technique (regression analysis). Use data mining to introduce greater efficiency into the audit process, without losing effectiveness.
EZ-R Stats, LLC
Session agenda - 1

Introduction and the Process for Managing the Business Risk of Fraud Introductions All Around Course Objectives Framework of risk management for fraud Fundamentals of data mining Data mining: The Engine That Drives analysis
Analytics and Regression

Sources of Analytics Data Basic and Intermediate ARTs
SAS 56 IIA Practice Advisory 2320 The Yellow Book (2007 revision) The Guide Managing the Business Risk of Fraud
Session Agenda (contd) Sampling refresher

Sampling The sampling process Sampling methods RAT-STATS

Random Numbers Determining Sample Size Case Study Attribute sampling Variable Sampling Case study Stratified Sampling Obtaining and Interpreting the results
Other Sampling Approaches DCAA Audit Package Sequential Sampling Overview of the process Attribute Sampling Variable Sampling
EZ-R Stats, LLC
Session Agenda (contd) Linear regression as an audit tool

Regression Analysis Overview Terms Statistical basis Charting Regression Seeing Is Believing Plotting Data

Inserting a Trend line Confidence Intervals Prediction Intervals Calculation of Statistical Confidence Bounds
Statistical Intervals
Case Study - Wake County Schools Bus Maintenance

Session Agenda (contd) Data mining, or How to test 100%

Overview Statistical Basis Data Conversion and Extraction Data mining objectives

Classification Trends Identification of extremes Major types of data analysis
Numeric Date Text
EZ-R Stats, LLC
Session Agenda (contd)
Excel as an Analytics tool

Macros Tools Data Analysis The Macro facility

Adding a little class to your audit VBA friend or foe
EZ-R Stats, LLC
Handout (CD)

CD with articles and software PowerPoint presentation More info at www.ezrstats.com
EZ-R Stats, LLC
Cockroach theory of auditing
If you spot one roach.
EZ-R Stats, LLC
Cockroach theory of auditing
There are probably 30 more that you dont see
EZ-R Stats, LLC
Statistics based roach hunting
Many frauds coulda/woulda/shoulda been detected with analytics

Overview
Fraud patterns detectable with digital analysis Basis for digital analysis approach Usage examples Continuous monitoring Business analytics
Objective 1
The Why and How

Three brief examples ACFE/IIA/AICPA Guidance Paper Practice Advisory 2320-1 Auditors Top 10 Process Overview Who, What, Why, When & Where
EZ-R Stats, LLC
Example 1 Wake County Transportation Fraud

Objective 1a
Supplier Kickback School Bus parts $5 million Jail sentences Period of years
EZ-R Stats, LLC
Objective 1a
Too little too late

Understaffed internal audit Software not used Data on multiple platforms Transaction volumes large
EZ-R Stats, LLC
Objective 1a
Preventable

Need structured, objective approach Let the data talk to you Need efficient and effective approach
EZ-R Stats, LLC
Objective 1
Regression Analysis
Stepwise to find relationships
Forwards Backwards Confidence Prediction
Intervals

EZ-R Stats, LLC
Objective 1
Data outliers

Sometimes an out and out Liar But how do you detect it?
EZ-R Stats, LLC
Objective 1
Data Outliers

Plot transportation costs vs. number of buses Drill down on costs
Preventive maintenance Fuel Inspection
EZ-R Stats, LLC
Scatter plot with prediction and confidence intervals
EZ-R Stats, LLC
Example 2
Objective 1a
Cost of six types of AIDS drugs

Total Cost of AIDS Drugs
200
Dollar Amount
150 100 50 0 NDC1 NDC2 NDC3 NDC4 NDC5 NDC6 Drug Type
NDC1 NDC2 NDC3 NDC4 NDC5 NDC6
EZ-R Stats, LLC
Objective 1
Medicare HIV Infusion Costs

CMS Report for 2005 South Florida - $2.2 Billion Rest of the country combined $.1 Billion
EZ-R Stats, LLC
Objective 1
Pareto Chart
Medicare HIV Infusion Costs - 2005 ($Billions) data source: HHS CMS
120.0% 100.0%
Annual Medicare Costs
80.0% 60.0% 40.0% 20.0% 0.0%

1 3 5 7 9 11 13 15
Pct Cum Pct
County
EZ-R Stats, LLC
Example 2
Objective 1a
Typical Prescription Patterns

AIDS Drugs Prescription Patterns
60.0
Dollar Value
50.0 40.0 30.0 20.0 10.0 0.0 Prov 1 Prov 2 Prov 3 Prov 4 Prov 5 Prov 6 Prescriber
NDC1 NDC2 NDC3 NDC4 NDC5 NDC6
EZ-R Stats, LLC
Example 2
Objective 1a
Prescriptions by Dr. X
Dr. X compared with Total Population
350 300 250 200 150 100 50 0 NDC1 NDC2 NDC3 NDC4 NDC5 NDC6 Drug Type
Dollar Amount
Population Dr. X
EZ-R Stats, LLC
Example 2
Off-label use
Objective 1a
Serostim

Treat wasting syndrome, side effect of AIDS, OR Used by body builders for recreational purposes One physician prescribed $11.5 million worth (12% of the entire state)
EZ-R Stats, LLC
Example 3
Revenue trends
Objective 1a
Overall Revenue Trend

1.2
Annual Billings
1.15 1.1 1.05 1 0.95 0.9 2001 2002 Calendar Year 2003 Overall Linear (Overall)
EZ-R Stats, LLC
Example 3
Dental Billings
Objective 1a
Rapid Increase in Revenues

5
Annual Billings ($millions)
4 3 2 1 0 2001 2002 Calendar Year 2003 Billings A Billings B Linear (Billings A)
EZ-R Stats, LLC
Objective 1b
Guidance Paper

A proposed implementation approach Managing the Business Risk of Fraud: A Practical Guide http://tinyurl.com/3ldfza Five Principles Fraud Detection Coordinated Investigation Approach
EZ-R Stats, LLC
Objective 1b
Managing the Business Risk of Fraud: A Practical Guide
ACFE, IIA and AICPA Exposure draft issued 11/2007, final 5/2008 Section 4 Fraud Detection
EZ-R Stats, LLC
Guidance Paper
Five Sections

Fraud Risk Governance Fraud Risk Assessment Fraud Prevention Fraud Detection Fraud Investigation and corrective action
EZ-R Stats, LLC
Risk Governance

Fraud risk management program Written policy managements expectations regarding managing fraud risk
EZ-R Stats, LLC
Risk Assessment

Periodic review and assessment of potential schemes and events Need to mitigate risk
EZ-R Stats, LLC
Fraud Prevention

Establish prevention techniques Mitigate possible impact on the organization
EZ-R Stats, LLC
Fraud Detection

Establish detection techniques for fraud Back stop where preventive measures fail, or Unmitigated risks are realized
EZ-R Stats, LLC
Fraud Investigation and Corrective Action

Reporting process to solicit input on fraud Coordinated approach to investigation Use of corrective action
EZ-R Stats, LLC
60 Minutes World of Trouble
2/15/09 Scott Pelley

Fraud Risk Governance one grand wink-wink, nod-nod Fraud Risk Assessment - categorically false Fraud Prevention my husband passed away Fraud Detection - We didn't know? Never saw one. Fraud Investigation and corrective action - Pick-APayment losses $36 billion
EZ-R Stats, LLC
Objective 1b
Section 4 Fraud Detection

Detective Controls Process Controls Anonymous Reporting Internal Auditing Proactive Fraud Detection
EZ-R Stats, LLC
Objective 1b
Proactive Fraud Detection
Data Analysis to identify: Anomalies Trends Risk indicators
EZ-R Stats, LLC
Fraud Detective Controls

Operate in the background Not evident in everyday business environment These techniques usually

Occur in ordinary course of business Corroboration using external information Automatically communicate deficiencies Use results to enhance other controls
Examples of detective controls

Whistleblower hot-lines (DHHS and OSA have them) Process controls (Medicaid audits and edits) Proactive fraud detection procedures

Data analysis Continuous monitoring Benfords Law
EZ-R Stats, LLC
Objective 1b
Specific Examples Cited

Journal entries suspicious transactions Identification of relationships Benfords Law Continuous monitoring
EZ-R Stats, LLC
Objective 1b
Data Analysis enhances ability to detect fraud

Identify hidden relationships Identify suspicious transactions Assess effectiveness of internal controls Monitor fraud threats Analyze millions of transactions
EZ-R Stats, LLC
Continuous Monitoring of Fraud Detection

Organization should develop ongoing monitoring and measurements Establish measurement criteria (and communicate to Board) Measurable criteria include:
EZ-R Stats, LLC
Measurable Criteria number of

fraud allegations fraud investigations resolved Employees attending annual ethics course Whistle blower allegations Messages supporting ethical behavior delivered by executives Vendors signing ethical behavior standards
Management ownership of each technique implemented
Each process owner should:

Evaluate effectiveness of technique regularly Adjust technique as required Document adjustments Report modifications needed for techniques which become less effective
EZ-R Stats, LLC
Practice Advisory 2320-1 Analysis and Evaluation

International standards for the professional practice of Internal Auditing Analytical audit procedures

Efficient and effective Useful in detecting
Differences that are not expected Potential errors Potential irregularities
EZ-R Stats, LLC
Analytical Audit Procedures
May include Study of relationships Comparison of amounts with similar information in the organization Comparison of amounts with similar information in the industry
EZ-R Stats, LLC
Analytical audit procedures

Performed using monetary amounts, physical quantities, ratios or percentages Ratio, trend and regression analysis Period to period comparisons Auditors should use analytical audit procedures in planning the engagement
EZ-R Stats, LLC
Factors to consider

Significance of the area being audited Assessment of risk Adequacy of system of internal control Availability and reliability of information Extent to which procedures provide support for engagement results
EZ-R Stats, LLC
Objective 1c
Peeling the Onion

Fraud Items
Possible Error Conditions Population as Whole
EZ-R Stats, LLC
Objective 1d
Fraud Pattern Detection

Round Numbers Market Basket Benfords Law
Stratification Target Group Trend Line
Gaps
Univariate
Holiday
Duplicates Day of Week
EZ-R Stats, LLC
Objective 1e
Digital Analysis (5W)

A little about the basics of digital analysis.
Who What Why Where When
Objective 1e
Who Uses Digital Analysis

Traditionally, IT specialists With appropriate tools, audit generalists (CAATs) Growing trend of business analytics Essential component of continuous monitoring
EZ-R Stats, LLC
Objective 1e
What - Digital Analysis
Using software to:
Classify Quantify Compare
Both numeric and non-numeric data
EZ-R Stats, LLC
Objective 1e
How - Assessing fraud risk

Basis is quantification Software can do the leg work Statistical measures of difference Chi square Kolmogorov-Smirnov D-statistic Specific approaches
EZ-R Stats, LLC
Objective 1e
Why - Advantages

Automated process Handle large data populations Objective, quantifiable metrics Can be part of continuous monitoring Can produce useful business analytics 100% testing is possible Quantify risk Repeatable process
EZ-R Stats, LLC
Objective 1e
Why - Disadvantages

Costly (time and software costs) Learning curve Requires specialized knowledge
EZ-R Stats, LLC
Objective 1e
When to Use Digital Analysis

Traditional intermittent (one off) Trend is to use it as often as possible Continuous monitoring Scheduled processing
EZ-R Stats, LLC
Objective 1e
Where Is It Applicable?
Any organization with data in digital format, and especially if:

Volumes are large Data structures are complex Potential for fraud exists
EZ-R Stats, LLC
Disadvantages of digital analysis
Cost

Software Training Skills not widely available Development costs Testing resources
Time consuming

EZ-R Stats, LLC
Objective 1
Objective 1 Summarized

Three brief examples CFE Guidance Paper Top 10 Metrics Process Overview Who, What, Why, When & Where
EZ-R Stats, LLC
Objective 1 - Summarized

Next is plan, perform

Statistical Sampling

Brief History / Timeline Overview Attribute Sampling Compliance Variable Sampling Numeric Estimates
EZ-R Stats, LLC
History of Sampling

Basis is two laws/theorems of probability Law of Large Numbers Central Limit Theorem
EZ-R Stats, LLC
Law of large numbers

Simulated rolling of dice
7 6 5 4 3 2 1 0 1 7 13 19 25 31 37 43 49 55 61 67 73 79 85 Observation Result Average Linear (Result)
Value
EZ-R Stats, LLC
Time Line - LLN

Indian mathematician Bramagupta 600 AD Italian mathematician Cardon 1500s Statement without proof that empirical statistics improve with more trials
EZ-R Stats, LLC
Time line LLN (continued)

Jacob Bernoulli first to prove in 1713 Foundation for central limit theorem
EZ-R Stats, LLC
Central limit theorem

Classic measure Mean of a sufficiently large number of random samples will be approximately normally distributed.
EZ-R Stats, LLC
The traditional explanation
EZ-R Stats, LLC
Central Limit Theorem

See it in action today Any population Large number of samples Average is normally distributed
EZ-R Stats, LLC
History of Central Limit Theorem

French mathematician Abraham de Moivre 1733 approximate distribution from tossing coin (heads/tails) Ho hum reaction French Mathematician LaPlace expanded it Ho hum reaction
History of CLT (contd)

Russian mathematician Lyapunov Proof in 1901 Same reaction
EZ-R Stats, LLC
Industrial revolution
Manufacturing
Engineering
Excitement!
EZ-R Stats, LLC
Students T
William Gosset - 1908 Guinness Brewery
EZ-R Stats, LLC
SAS 39

Effective June, 1983 Exposure draft for revision in 2009
EZ-R Stats, LLC
Attribute sampling

Buonaccorsi (1987) Refined calculations Few software packages use it
EZ-R Stats, LLC
Overview

Sample size calculations Attribute sampling Variable sampling Random number generators
EZ-R Stats, LLC
Sample size calculation

Its a guess Every package different answer Need to know the population But thats why youre taking a sample!
EZ-R Stats, LLC
Attribute Sampling
Using RAT-STATS
Unrestricted populations
Managing the business risk of fraud using sampling and data mining
EZ-R Stats, LLC
Session Objectives
1.
2. 3. 4.
Understand what is attribute sampling and when to use it Understand unrestricted populations Overview of the process using RAT-STATS Understand the formula behind the computations
EZ-R Stats, LLC
Attribute sampling
Attribute Compliance
testing Signatures on approval documents, attachment of supporting documentation, etc.
EZ-R Stats, LLC
Statistical approach
Recommended Economical Efficient Requires
determination of a sample size

Overview of process
Determine the sampling objective

Confidence Precision
Determine required sample size Identify samples to be selected based upon random numbers Pull the sample and test Compute the sampling results (i.e. estimate of range)
How this is done in RAT-STATS

The sampling parameters are first developed by the auditor RAT-STATS is used to compute sample size RAT-STATS used to generate random numbers Pull the sample and test Enter results in RAT-STATS to compute estimates
Step 1 Develop sampling parameters

1. 2. 3. 4.
Size of population Expected error rate Required confidence Required precision
EZ-R Stats, LLC
Step 2 Obtain the random numbers

Done by entering info into RAT-STATS Output can be a variety of sources:

Text File Excel Microsoft Access Print File
EZ-R Stats, LLC
Step 3 Pull the sample

Each random number selected corresponds with an item Put the selected item on a separate schedule
EZ-R Stats, LLC
Step 4 - Test each selected item
Generally requires reviewing documents
EZ-R Stats, LLC
Step 5 Compute the results

Enter summary information into RAT-STATS Output can be in a variety of formats:

Excel Microsoft Access Text File Print File Printer
EZ-R Stats, LLC
Thats It!

Now well see an actual demo using the RAT-STATS software Excel population of 5,000 invoices Results of test of attributes stored in the worksheet
EZ-R Stats, LLC
Variable Sampling
Using RAT-STATS
Unrestricted populations
EZ-R Stats, LLC
Session Objectives
1.
2.
3.
4.
Understand what variable sampling is and when to use it Understand unrestricted populations Overview of the process using RATSTATS Understand the formula behind the computations
Variable sampling
Variable Estimating
account balances Estimating transaction totals
EZ-R Stats, LLC
Statistical approach
Recommended Economical Efficient Requires
determination of a sample size

Overview of process


1. 2. 3.
Probe sample Statistical measure Excel formula
EZ-R Stats, LLC
1. 2. 3.
Size of population Average value Standard deviation
EZ-R Stats, LLC
Step 2 Obtain the random numbers Done by entering info into RAT-STATS Output can be a variety of sources: Text File Excel Microsoft Access Print File

Each
random number selected corresponds with an item Put the selected item on a separate schedule
EZ-R Stats, LLC

Generally
requires reviewing documents Example data contains both examined and audited value.
EZ-R Stats, LLC


EZ-R Stats, LLC
Thats It!
Now
well see an actual demo using the RAT-STATS software Excel population of 5,000 invoices Audited values stored in the worksheet
EZ-R Stats, LLC
Attribute Sampling
Using RAT-STATS
Stratified populations
EZ-R Stats, LLC
Session Objectives
1.
2.
Understand what is stratification and when to use it Overview of the process using RAT-STATS
EZ-R Stats, LLC
Stratified sampling
Strata Homogenous More
efficient in some instances
EZ-R Stats, LLC
Overview of process

Separation into strata Determine the sampling objective

EZ-R Stats, LLC


1. 2. 3. 4.
Size of population Expected error rate Required confidence Required precision
EZ-R Stats, LLC

Done by entering info into RAT-STATS Output can be a variety of sources:

Text File Excel Microsoft Access Print File
EZ-R Stats, LLC

Each random number selected corresponds with an item Put the selected item on a separate schedule
EZ-R Stats, LLC
Generally requires reviewing documents
EZ-R Stats, LLC


EZ-R Stats, LLC
Thats It!

Now well see an actual demo using the RAT-STATS software Excel population of 5,000 invoices Results of test of attributes stored in the worksheet
EZ-R Stats, LLC
Variable Sampling
Using RAT-STATS
Stratified populations
EZ-R Stats, LLC
Session Objectives
1.
2.
3.
4.
Understand what stratified sampling is and when to use it Populations benefiting from stratified sampling Overview of the process using RATSTATS Understand the formula behind the computationsof fraud Managing the business risk EZ-R Stats, LLC
Stratified variable sampling

Stratified Variable Estimating
amounts Narrower standard deviation
EZ-R Stats, LLC
Overview of process


1. 2. 3.
Probe sample Statistical measure Excel formula
EZ-R Stats, LLC
1. 2. 3.
4.
Number of strata Size of population Average value Standard deviation


Done by entering info into RAT-STATS Multi-stage random numbers Output can be a variety of sources: Text File Excel Microsoft Access Print File

Each
random number selected corresponds with an item in a strata Put the selected item on a separate schedule
EZ-R Stats, LLC

Generally
requires reviewing documents Example data contains both examined and audited value.
EZ-R Stats, LLC


EZ-R Stats, LLC
Thats It!
Now
well see an actual demo using the RAT-STATS software Excel population of 5,000 invoices Divided into three strata Audited values stored in the worksheet

Next is cost reduction

Techniques for cost reduction
Optimize sample size (most bang for the buck) Skip sampling review 100% of transactions using computer assisted audit techniques (CAATs)
EZ-R Stats, LLC
Sample optimization
Sequential sampling
EZ-R Stats, LLC
University of Hawaii
Banana aphids
EZ-R Stats, LLC
Sequential sampling
Banana aphids
EZ-R Stats, LLC
100% test using CAATs

Provides complete coverage Best practice Basis for continuous monitoring Repeatable process
EZ-R Stats, LLC

Next is Yellow Book and SAS 56

Yellow book standards

Standards regarding statistical sampling and IT
EZ-R Stats, LLC
General standards
3.43 Technical Knowledge and competence
The staff assigned to conduct an audit or attestation engagement under GAGAS must collectively possess the technical knowledge, skills, and experience necessary to be competent for the type of work being performed before beginning work on that assignment. The staff assigned to a GAGAS audit or attestation engagement should collectively possess:
Stat sampling and IT

Skills appropriate for the work being performed. For example, staff or specialist skills in
(1) statistical sampling if the work involves use of statistical sampling; (2) information technology
SAS 56 Analytical procedures

Requires use of analytic review procedures for: Audit planning Overall review stages
EZ-R Stats, LLC
SAS 56 Analytical Review

Encourages use of analytical review Provides guidance

A wide variety of analytical procedures may be useful for this purpose.
EZ-R Stats, LLC

Next is linear regression

Next Metric
1. 2. 3. 4. 5. 6. 7. 8. 9. 10.
Outliers Stratification Day of Week Round Numbers Made Up Numbers Market basket Trends Gaps Duplicates Dates
EZ-R Stats, LLC
7 - Trends
Trend Busters
Does the pattern make sense?
ACME Technology
30,000 25,000 20,000 15,000 10,000 5,000 0
-0 7 M ay -0 7 Ju l-0 Se 7 p07 N ov -0 7 Ja n08 M ar -0 M 8 ay -0 8 n0 7 ar
Amount
Sales Employee Count
Ja
Date
EZ-R Stats, LLC
7 Trends
Trend Busters

Linear regression Sales are up, but cost of goods sold is down Spikes
EZ-R Stats, LLC
7 Trends
Purpose / Type of Errors

Identify trend lines, slopes, etc. Correlate trends Identify anomalies Key punch errors where amount is order of magnitude
EZ-R Stats, LLC
7 Trends
Linear Regression
Test
relationships (e.g. invoice amount and sales tax) Perform multi-variable analysis
7 Trends
How is it done?

Estimate linear trends using best fit Measure variability (standard errors) Measure slope Sort descending by slope, variability, etc.
EZ-R Stats, LLC
7 Trends Trend Lines by Account - Example Results
Account 32451 43517 32451 43517 43870 54630
N 18 17 27 32 23 56
Slope 1.230 1.070 1.023 1.010 0.340 -0.560
Std Err 0.87 4.3 0.85 0.36 2.36 1.89
Generally the trend is gently sloping up, but two accounts (43870 and 54630) are different.
Scatter plot with prediction and confidence intervals
EZ-R Stats, LLC

Next is data mining

Objective 6
Basis for Pattern Detection

Analytical review Isolate the significant few Detection of errors Quantified approach
EZ-R Stats, LLC
Objective 2
Understanding the Basis

Quantified Approach Population vs. Groups Measuring the Difference Stat 101 Counts, Totals, Chi Square and K-S The metrics used
EZ-R Stats, LLC
Objective 2a
Quantified Approach
Based on measureable differences Population vs. Group Shotgun technique
EZ-R Stats, LLC
Objective 2a
Detection of Fraud Characteristics
Something is different than expected
EZ-R Stats, LLC
Objective 2b
Fraud patterns
Common theme something is different Groups Group pattern is different than overall population
EZ-R Stats, LLC
Objective 2c
Measurement Basis
Transaction
counts Transaction amounts
EZ-R Stats, LLC
Objective 2d
A few words about statistics (the s word)

Detailed knowledge of statistics not necessary Software packages do the numbercrunching Statistics used only to highlight potential errors/frauds Not used for quantification
EZ-R Stats, LLC
Objective 2d
How is digital analysis done?
Comparison of group with population as a whole Can be based on either counts or amounts Difference is measured Groups can then be ranked using a selected measure High difference = possible error/fraud
EZ-R Stats, LLC
Demo in Excel of the process

Based roughly on the Wake County Transportation fraud Illustrates how the process works, using Excel
EZ-R Stats, LLC
Objective 2d
Histograms

Attributes tallied and categorized into bins Counts or sums of amounts
EZ-R Stats, LLC
Objective 2d
Two histograms obtained
Population and group

Group
80 70 60 50 40 30 20 10 0
Population
700 600 500 400 300 200 100 0 Jan- Feb- Mar- Apr- May- Jun- Jul- Aug- Sep- Oct- Nov- Dec07 07 07 07 07 07 07 07 07 07 07 07
Jan- Feb- Mar- Apr- May- Jun- Jul- Aug- Sep- Oct- Nov- Dec07 07 07 07 07 07 07 07 07 07 07 07
EZ-R Stats, LLC
Objective 2d
Compute Cumulative Amount for each
Count by Month
80 120.0% 70 60 50
Count
Cum Pct
100.0% 80.0%
40 30 20 10 0
Ja n0 Fe 7 bM 07 ar -0 Ap 7 r-0 M 7 ay -0 Ju 7 n0 Ju 7 lAu 07 g0 Se 7 p0 O 7 ct0 No 7 v0 De 7 c07
60.0%
40.0% 20.0%
0.0%
Ja n07 Ju l-0 7 07 Se p07 M ay M ar ov -0 7 N 07
Month
EZ-R Stats, LLC
Objective 2d
Are the histograms different?

Two statistical measures of difference Chi Squared (counts) K-S (distribution) Both yield a difference metric
EZ-R Stats, LLC
Objective 2d
Chi Squared
Classic test on data in a table Answers the question are the rows/columns different Some limitations on when it can be applied
EZ-R Stats, LLC
Objective 2d
Chi Squared

Table of Counts Degrees of Freedom Chi Squared Value P-statistic Computationally intensive
EZ-R Stats, LLC
Objective 2d
Kolmogorov-Smirnov
Two Russian mathematicians Comparison of distributions Metric is the d-statistic
Objective 2d
How is K-S test done?
Four step process

1.
2.
3.
4.
For each cluster element determine percentage Then calculate cumulative percentage Compare the differences in cumulative percentages Identify the largest difference
EZ-R Stats, LLC
Objective 2d - KS
Kolmogorov-Smirnov
EZ-R Stats, LLC
Objective 2e
Classification by metrics
Stratification Day of week Happens on holiday Round numbers Variability Benfords Law Trend lines Relationships (market basket) Gaps Duplicates
EZ-R Stats, LLC
Objective e
Auditors Top 10 Metrics

1. 2. 3. 4. 5. 6.
7.
8. 9. 10.
Outliers / Variability Stratification Day of Week Round Numbers Made Up Numbers Market basket Trends Gaps Duplicates Dates
EZ-R Stats, LLC
Objective 2
Understanding the Basis

Quantified Approach Population vs. Groups Measuring the Difference Stat 101 Counts, Totals, Chi Square and K-S The metrics used
EZ-R Stats, LLC
1. 2. 3. 4. 5.
Understand why and how Understand statistical basis for quantifying differences Identify ten general tools and techniques Understand examples done using Excel How pattern detection fits in
Next are the metrics
EZ-R Stats, LLC
Its that time!
Session Break!
EZ-R Stats, LLC
Objective 3
The Top 10 Metrics

Overview Explain Each Metric Examples of what it can detect How to assess results
EZ-R Stats, LLC
Objective 3
Trapping anomalies
EZ-R Stats, LLC
Objective 3
Fraud Pattern Detection

Round Numbers Market Basket Benfords Law
Stratification Target Group Trend Line
Gaps
Univariate
Holiday
Duplicates Day of Week
EZ-R Stats, LLC
1 - Outliers
Outliers / Variability
Outliers are amounts which are significantly different from the rest of the population
1 - Outliers
Outliers / Variability

Charting (visual) Software to analyze z-scores Top and Bottom 10, 20 etc. High and low variability (coefficient of variation)
EZ-R Stats, LLC
1 - Outliers
Drill down to the group level
Basic statistics Minimum, maximum and average Variability Sort by statistic of interest Variability (coefficient of variation) Maximum, etc.
EZ-R Stats, LLC
1 - Outliers
Example Results
Provider
3478421 2356721 3546789 5463122
N
3,243 4,536 3,421 2,311
Coeff Var
342.23 87.23 23.25 18.54
Two providers (3478421 and 2356721) had significantly more variability in the amounts of their claims than all the rest.
Next Metric
1. 2. 3.
4.
5. 6. 7.
8.
9. 10.
EZ-R Stats, LLC
2 - Stratification
Unusual stratification patterns
Do you know how your data looks?

2 - Stratification
Stratification - How
Charting (visual) Chi Squared Kolmogorov-Smirnov By groups
EZ-R Stats, LLC
2 Stratification
Purpose / types of errors

Transactions out of the ordinary Up-coding insurance claims Skewed groupings Based on either count or amount
EZ-R Stats, LLC
2 Stratification
The process?
1. 2. 3. 4. 5.
Stratify the entire population into bins specified by auditor Same stratification on each group (e.g. vendor) Compare the group tested to the population Obtain measure of difference for each group Sort descending on difference measure
EZ-R Stats, LLC
2 Stratification
Units of Service Stratified Example Results

Provider 2735211 N 6,011 Chi Sq 7,453 D-stat 0.8453
4562134
4321089 4237869
8,913
3,410 2,503
5,234
342 298
0.7453
0.5231 0.4632
Two providers (2735211 and 4562134) are shown to be much different from the overall population (as measured by Chi Square).
Next Metric
1. 2. 3. 4. 5. 6. 7. 8. 9. 10.
EZ-R Stats, LLC
3 Day of Week
Day of Week

Activity on weekdays Activity on weekends Peak activity mid to late week
EZ-R Stats, LLC
3 Day of Week
Identify unusually high/low activity on one or more days of week Dentist who only handled Medicaid on Tuesday Office is empty on Friday
EZ-R Stats, LLC
How it is done?

Programmatically check entire population Obtain counts and sums by day of week (1-7) Prepare histogram For each group do the same procedure Compare the two histograms Sort descending by metric (chi square/dstat)
EZ-R Stats, LLC
3 Day of Week
Day of Week - Example Results
Provider 2735211 4562134 4321089 4237869
N 5,404 5,182 5,162 7,905
Chi Sq 12,435 7,746 87 56
D-stat 0.9802 0.8472 0.321 0.2189
Provider 2735211 only provided service for Medicaid on Tuesdays. Provider 4562134 was closed on Thursdays and Fridays.
Next Metric
1.
2. 3. 4. 5. 6. 7. 8. 9. 10.
EZ-R Stats, LLC
4 Round Numbers
Round Numbers
Its about.
Estimates!
EZ-R Stats, LLC
4 Round Numbers

Isolate estimates Highlight account numbers in journal entries with round numbers Split purchases (under the radar) Which groups have the most estimates
EZ-R Stats, LLC
4 Round Numbers
Round numbers
Classify population amounts $1,375.23 is not round $5,000 is a round number type 3 (3 zeros) $10,200 is a round number type 2 (2 zeros) Quantify expected vs. actual (d-statistic) Generally represents an estimate Journal entries
EZ-R Stats, LLC
4 Round Numbers
Round Numbers in Journal Entries - Example Results

Account
2735211 4562134 4321089 4237869
N
4,136 833 8,318 9,549
Chi Sq
54,637 35,324 768 546
D-stat
0.9802 0.97023 0.321 0.2189
Two accounts, 2735211 and 4562134 have significantly more round number postings than any other posting account in the journal entries.
Next Metric
1.
2. 3. 4.
5.
6. 7. 8.
9.
10.
EZ-R Stats, LLC
5 Made up numbers
Made up Numbers
Curb stoning Imaginary numbers Benfords Law

5 Made Up Numbers
What can be detected
Made up numbers e.g. falsified inventory counts, tax return schedules
EZ-R Stats, LLC
5 Made Up Numbers
Benfords Law using Excel

Basic formula is =log(1+(1/N)) Workbook with formulae available at http://tinyurl.com/4vmcfs Obtain leading digits using Left function, e.g. left(Cell,1)
EZ-R Stats, LLC
5 Made Up Numbers
Made up numbers

Benfords Law Check Chi Square and d-statistic First 1,2,3 digits Last 1,2 digits Second digit Sources for more info
EZ-R Stats, LLC
5 Made Up Numbers
How is it done?
Decide type of test (first 1-3 digits, last 1-2 digit etc) For each group, count number of observations for each digit pattern Prepare histogram Based on total count, compute expected values For the group, compute Chi Square and d-stat Sort descending by metric (chi square/dstat)
EZ-R Stats, LLC
5 Made Up Numbers Invoice Amounts tested with Benfords law - Example Results
Store 324 563 432 Hi Digit 79 89 23 Chi Sq 5,234 4,735 476 D-stat 0.9802 0.97023 0.321
217
74
312
0.2189
During tests of invoices by store, two stores, 324 and 563 have significantly more differences than any other store as measured by Benfords Law.
EZ-R Stats, LLC
Next Metric
1. 2. 3. 4. 5. 6. 7. 8. 9. 10.
EZ-R Stats, LLC
6 Market Basket
Market Basket
Medical Ping ponging Pattern associations Apriori program References at end of slides Apriori Latin a (from) priori (former) Deduction from the known
EZ-R Stats, LLC
6 Market basket

Unexpected patterns and associations Based on market basket concept Unusual combinations of diagnosis code on medical insurance claim
EZ-R Stats, LLC
6 Market basket
Market Basket
JE Accounts JE Approvals Credit card fraud in Japan taxi and ATM
EZ-R Stats, LLC
6 Market basket
How is it done?

First, identify groups, e.g. all medical providers for a patient Next, for each provider, assign a unique integer value Create a text file containing the values Run apriori analysis
EZ-R Stats, LLC
6 Market basket
Apriori outputs

For each unique value, probability of other values If you see Dr. Jones, you will also see Dr. Smith (80% probability) If you see a JE to account ABC, there will also an entry to account XYZ (30%)
EZ-R Stats, LLC
Next Metric
1. 2. 3. 4. 5. 6. 7. 8. 9. 10.
EZ-R Stats, LLC
8 - Gaps
Numeric Sequence Gaps
Whats there is interesting, whats not there is critical

8 Gaps

Missing documents (sales, cash, etc.) Inventory losses (missing receiving reports) Items that walked off
EZ-R Stats, LLC
8 Gaps
How is it done?

Check any sequence of numbers supposed to be complete, e.g. Cash receipts Sales slips Purchase orders
EZ-R Stats, LLC
8 Gaps
Gaps Using Excel

Excel sort and check Excel formula Sequential numbers and dates
EZ-R Stats, LLC
8 Gaps
Gap Testing - Example Results
Start 10789 12523 17546
End 10791 12526 17548
Missing 1 2 1
Four check numbers are missing.
EZ-R Stats, LLC
Next Metric
1. 2.
3.
4. 5. 6.
7.
8. 9. 10.
EZ-R Stats, LLC
9 - Duplicates
Duplicates
Why is there more than one?

Same, Same, Same, and
Same, Same, Different

9 Duplicates
Two types of (related) tests
Same items same vendor, same invoice number, same invoice date, same amount Different items same employee name, same city, different social security number
EZ-R Stats, LLC
9 - Duplicates
Duplicate Payments
High payback area
Fuzzy
logic
Overriding software controls
EZ-R Stats, LLC
Fuzzy matching with software

9 - Duplicates
Levenshtein distance Soundex Like clause in SQL Regular expression testing in SQL Vendor/employee situations
Russian physicist
EZ-R Stats, LLC
9 - Duplicates
How is it done?

First, sort file in sequence for testing Compare items in consecutive rows Extract exceptions for follow-up
EZ-R Stats, LLC
9 - Duplicates
Possible Duplicates - Example Results
Vendor 10245 10245 17546
Invoice Date 6/15/2007 8/31/2007 2/12/2007
Invoice Amount 3,544.78 2,010.37 1,500.00
Count 4 2 2
Five invoices may be duplicates.

Next Metric
1.
2. 3. 4.
5.
6. 7. 8.
9.
10.
EZ-R Stats, LLC
10 - Dates
Date Checking
If were closed, why is there

Adjusting journal entry? Receiving report?
Payment issued?
10 Dates
Holiday Date Testing
Red Flag indicator
EZ-R Stats, LLC
Date Testing challenges

10 Dates
Difficult to determine Floating holidays Friday, Saturday, Sunday, Monday
EZ-R Stats, LLC
10 Dates
Typical audit areas

Journal entries Employee expense reports Business telephone calls Invoices Receiving reports Purchase orders
EZ-R Stats, LLC
10 Dates
Determination of Dates

Transactions when business is closed Federal Office of Budget Management An excellent fraud indicator in some cases
EZ-R Stats, LLC
10 Dates
Holiday Date Testing
Identifying holiday dates: Error prone Tedious U.S. only
EZ-R Stats, LLC
10 Dates
Federal Holidays

Established by Law Ten dates Specific date (unless weekend), OR Floating holiday
EZ-R Stats, LLC
10 Dates
Federal Holiday Schedule

Office of Personnel Management Example of specific date Independence Day, July 4th (unless weekend) Example of floating date Martin Luther Kings birthday (3rd Monday in January) Floating Thanksgiving 4th Thursday in November
EZ-R Stats, LLC
10 Dates
How it is done?

Programmatically count holidays for entire population For each group, count holidays Compare the two histograms (group and population) Sort descending by metric (chi square/d-stat)
EZ-R Stats, LLC
10 Dates
Holiday Counts - Example Results
Employee Number 10245 32325 17546
N 37 23 18
Chi Sq 5,234 4,735 476
D-stat 0.9802 0.97023 0.321
24135
34
312
0.2189
Two employees (10245 and 32325) were off the chart in terms of expense amounts incurred on a Federal Holiday.
Objective 3
The Top 10 Metrics

Overview Explain Each Metric Examples of what it can detect How to assess results
EZ-R Stats, LLC
1. 2. 3. 4. 5.
Understand why and how Understand statistical basis for quantifying differences Identify ten general tools and techniques Understand examples done using Excel How pattern detection fits in
Next using Excel

Objective 4
Use of Excel

Built-in functions Add-ins Macros Database access
EZ-R Stats, LLC
Objective 4
Excel templates
Variety of tests

Round numbers Benfords Law Outliers Etc.
EZ-R Stats, LLC
Objective 4
Excel Univariate statistics
Work with Ranges =sum, =average, =stdevp =largest(Range,1), =smallest(Range,1) =min, =max, =count Tools | Data Analysis | Descriptive Statistics
EZ-R Stats, LLC
Objective 4
Excel Histograms

Tools | Data Analysis | Histogram Bin Range Data Range
EZ-R Stats, LLC
Objective 4
Excel Gaps testing
Sort by sequential value =if(thiscell-lastcell <> 1,thiscell-lastcell,0) Copy/paste special Sort
EZ-R Stats, LLC
Objective 4
Detecting duplicates with Excel

Sort by sort values =if testing =if(=and(thiscell=lastcell, etc.))
EZ-R Stats, LLC
Objective 4
Performing audit tests with macros

Repeatable process Audit standardization Learning curve Streamlining of tests More efficient and effective Examples http://ezrstats.com/Macros/home.html
EZ-R Stats, LLC
Objective 4
Using database audit software
Many built-in functions right off the shelf with SQL Control totals Exception identification Drill down Quantification June 2008 article in the EDP Audit & Control Journal (EDPACS) SQL as an audit tool
http://ezrstats.com/doc/SQL_As_An_Audit_Tool.pdf
EZ-R Stats, LLC
Objective 4
Use of Excel
Built-in functions Add-ins Macros Database access
EZ-R Stats, LLC
1. 2. 3. 4.
5.
Understand why and how Understand statistical basis for quantifying differences Identify ten general tools and techniques Understand examples done using Excel How Pattern Detection fits in
Next Fit
Objective 5
How Pattern Detection Fits In
Business Analytics Fraud Pattern Detection Continuous monitoring
EZ-R Stats, LLC
Objective 5
Where does Fraud Pattern Detection fit in?
Right in the middle

Business Analytics Fraud Pattern Detection Continuous fraud pattern detection Continuous Monitoring
EZ-R Stats, LLC
Objective 5
Business Analytics

Fraud analytics -> business analytics Business analytics -> fraud analytics
EZ-R Stats, LLC
Objective 5
Role in Continuous Monitoring (CM)
Fraud analytics can feed (CM) Continuous fraud pattern detection Use output from CM to tune fraud pattern detection
EZ-R Stats, LLC

EZ-R Stats, LLC
Links for more information

Kolmogorov-Smirnov http://tinyurl.com/y49sec Benfords Law http://tinyurl.com/3qapzu Chi Square tests http://tinyurl.com/43nkdh Continuous monitoring http://tinyurl.com/3pltdl
EZ-R Stats, LLC
Market Basket

Apriori testing for ping ponging Temple University http://tinyurl.com/5vax7r Apriori program (open source) http://tinyurl.com/5qehd5 Article Medical ping ponging http://tinyurl.com/5pzbh4
EZ-R Stats, LLC
Excel macros used in auditing
Excel as an audit software http://tinyurl.com/6h3ye7 Selected macros http://ezrstats.com/Macros/home.html
Spreadsheets forever http://tinyurl.com/5ppl7t
EZ-R Stats, LLC
Questions?
EZ-R Stats, LLC
Contact info
Phone: E-mail:
(919)-219-1622
Mike.Blakley@ezrstats.com Blog: http://blog.ezrstats.com
EZ-R Stats, LLC

Managing The Business Risk of Fraud

Uploaded by

Copyright:

Available Formats

Managing The Business Risk of Fraud

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Managing The Business Risk of Fraud

Uploaded by

Copyright:

Available Formats

Managing the Business Risk of Fraud using Sampling and Data Mining

PWC Global Survey Nov, 2009

meet performance targets keep their jobs keep access to funding

Managing the business risk of fraud

EZ-R Stats, LLC

Managing the business risk of fraud

EZ-R Stats, LLC

Analytics and Regression

Sources of Analytics Data Basic and Intermediate ARTs

Session Agenda (contd) Sampling refresher

Sampling The sampling process Sampling methods RAT-STATS

Managing the business risk of fraud

EZ-R Stats, LLC

Session Agenda (contd) Linear regression as an audit tool

Case Study - Wake County Schools Bus Maintenance

Session Agenda (contd) Data mining, or How to test 100%

Classification Trends Identification of extremes Major types of data analysis

Numeric Date Text

Managing the business risk of fraud

EZ-R Stats, LLC

Session Agenda (contd)

Excel as an Analytics tool

Macros Tools Data Analysis The Macro facility

Adding a little class to your audit VBA friend or foe

Managing the business risk of fraud

EZ-R Stats, LLC

CD with articles and software PowerPoint presentation More info at www.ezrstats.com

Managing the business risk of fraud

EZ-R Stats, LLC

Cockroach theory of auditing

If you spot one roach.

Managing the business risk of fraud

EZ-R Stats, LLC

Cockroach theory of auditing

There are probably 30 more that you dont see

Managing the business risk of fraud

EZ-R Stats, LLC

Statistics based roach hunting

Many frauds coulda/woulda/shoulda been detected with analytics

Managing the business risk of fraud EZ-R Stats, LLC

The Why and How

Managing the business risk of fraud

EZ-R Stats, LLC

Example 1 Wake County Transportation Fraud

Managing the business risk of fraud

EZ-R Stats, LLC

Too little too late

Managing the business risk of fraud

EZ-R Stats, LLC

Managing the business risk of fraud

EZ-R Stats, LLC

Stepwise to find relationships

Forwards Backwards Confidence Prediction

Managing the business risk of fraud

EZ-R Stats, LLC

Managing the business risk of fraud

EZ-R Stats, LLC

Plot transportation costs vs. number of buses Drill down on costs

Preventive maintenance Fuel Inspection