NOT

How to Lie with Benefit-Cost Analysis

Scott Farrow, UMBC

Analyzing public decisions using benefit-cost is anathema for some, a truth nostrum for others

and a targeted diagnostic for a few (heavily concentrated among economists). Its surface

simplicity and its subtle complexities make it easy to lie, accidentally or not. Conversely,

discerning what truth there may be about public policy is aided by warning signs of lying and

abetted by signs of how not to lie.

Benefit-cost analysis in a simple form was described by Benjamin Franklin, evolved by

economists, required by Presidents of both parties to analyze government regulations and

advocated for use on a broader basis primarily by Republicans. It’s used by charities such as the

Robin Hood Foundation in New York to help allocate charitable giving and by the World Bank

to inform the international allocation of funding. Governments issue guidelines on its use and

textbooks struggle to explain how to do it right but don’t provide the vivid examples of doing it

wrong that are so hard to forget. Most of us end up hearing fragments of studies as sound bites

about how good or bad the “bottom line” is for some program. What is benefit-cost analysis

doing and why is it so hard to avoid lying?

Benefit-cost analysis tries to answer whether a society is better off by taking an action when

impacts of all kinds are considered, including those outside the marketplace. Dollar values (in

the U.S.) are used to weight the impacts such as reduced building damages or deaths and these

impacts are added up “to whomsoever they accrue”. The bottom line measure, the eponymous

benefits less the costs (net benefit) is defined as a measure of economic efficiency (and who can

be against efficiency?). A positive value is standardly interpreted to mean that society will be

1
better off by taking the action (the subtler complexity is in Lie #4 below). This apparent

simplicity obscures numerous challenges in practice and interpretation so in this political season

I offer the following lies and how to avoid them as aids to distinguish among good, bad and ugly

analyses.

Lie #1: Be selective in your impacts and values

Start easy with this no-brainer tactic: Include impacts that slant the benefit or cost your way and

dispute or ignore those that tilt the other way. If you want to reduce the benefits of regulations to

reduce oil spills, exclude those who would pay a small bit more to avoid such events even if they

never visit the Gulf or Alaska (officially, non-use value). If a rule about the cleanliness of

drinking water also affects how much to clean up old industrial sites and you want to reduce

costs; ignore the effect on cleaning up old industrial sites

How not to lie: Include all the impacts and values for which credible (ah, there’s the rub)

estimates exist and which seem potentially large enough to change an opinion. No one is

prescient enough nor are there enough data to include everything, but see that the core elements

are included in a responsible way.

What you can do. Ask yourself “Are there major elements missing, or too many present in this

analysis?” This is perhaps the most challenging detective work, can you find the impact that

didn’t bark in the night and didn’t make it into the report?

Lie #2: Confuse the baseline

Choose a comparison that makes your desired impacts larger and your costs smaller, perhaps

even by choosing different baselines for benefits and costs. Impacts only exist if there is a

2
change from some starting point, the baseline. You see this when political Party A touts its

impact starting from a low point of the business cycle and political Party B responds measuring

from the high point of the cycle (or vice versa). Or build an inflation factor into costs but not

benefits (or vice versa). Or ignore history that States or industry or technology seem to be

moving over time and assume all factors will stay constant if that helps you.

How not to lie: Be consistent and clear about the baseline, including whether factors are

changing over time. The standard baseline is in comparison to doing nothing (the status quo) but

economies evolve in complex ways so a baseline that evolves over time is ok.

What you can do: Ask yourself “What is the basis of comparison? Is that reasonable and is it the

same for both benefits and costs?

Lie #3: Count jobs entirely as a benefit

After you compute the benefits of the policy (typically by looking at benefits received by all of

consumers, industry, and changes in government revenue), also count the number of employees

and the amount they will be paid as a benefit. After all, labor is about two-thirds of all costs in

the US and so if you can count two-thirds of the cost as a benefit, then that gets you a long ways

toward a positive net benefit. A variation on this is to count as a benefit the new jobs in your

local region while ignoring the loss or shift in jobs from another region. This is politically

correct from either party…go for it!

How not to lie: Decide whether unemployment rates are normal or high. If they are normal,

then there is no additional benefit from new jobs as your new job is just taking labor effort away

from some other job, it washes out. Recent times have seen unusually high rates of

unemployment which does create a justification for a partial benefit from new employment. It’s

3
partial because it depends on the wage a person requires to work. If a person would work for

next to nothing, then the whole wage is a benefit; if they won’t work for anything less than a

multiple of the minimum wage, then the benefit is the wage less the amount for which they will

choose to work. A back of the envelope approach on who will work for how much suggests

about fifty percent of the wage as a benefit in times of high unemployment.

What you can do: Ask yourself, “Will this policy be implemented during full employment or

high unemployment?” and “Are the new jobs just being taken away from another location that is

included in our calculations?”

Lie #4: Cite the bottom line as a crystal clear measure of improvement.

Find a report with a positive net value for the project or policy and tout it as quantitative support

that it is better for all. Corollary, find a negative value and state it’s bad for society. Don’t

mention caveats, they are such tiresome things and time is short.

How not lie: Be cautious about your conclusion. A positive value is like a green light on a

larger set of dashboard instruments. Society will unambiguously be better off only if

compensation is paid to all who incur costs which bears its own problems; or if the value of an

additional dollar is the same to all people, whether rich or poor, and society values equally a

dollar going to any individual. Falling off this unlikely knife edge leaves ambiguous the bottom

line of a single benefit-cost analysis. Treating everyone equally is a reasonable place to start and

the professional standard but it is not always the end of the story.

What you can do: Ask yourself, “Am I comfortable with adding up everyone’s impacts no

matter who they are on this issue?” “Is there evidence on who is impacted that is re-incorporated

into the analysis?”

4
Lie #5: Act as if a number is certain

Be confident, be very very confident. Report impacts in small fractions, report dollar values to

some small amount; hundreds or tens of dollars, or heh, go all the way and report the total in

cents. Don’t worry about conveying uncertainty about the number and don’t get bogged down in

defining whether your number is formally an average response (the mean).

How not to lie: Convey some measure of the accuracy of your measurements. Ideally this is

reflected in the number of significant digits (no, that’s not how many nails and toes have polish)

such as whether your data are precise only to the millions or by reporting statistical measures of

dispersion such as a standard error. Carry out “what if” or sensitivity analysis or in today’s easy

to computer world, do a “Monte Carlo” simulation which is like doing hundreds or thousands of

“what if” analyses.

What you can do. Ask “Does there seem to be a false level of precision in this analysis?” and

“Are we told whether the results change if reasonable changes are made to the analysis?”

Lie #6: There are no professional ethics

Provide the number, any way you can, that you think your boss or your client is expecting;

sometimes paraphrased as “I can get you any number you want, what do you want it to be?”

How not to lie: Even if there is no Hippocratic Oath for economists or policy analysts, there ar e

professionally acceptable ways of doing things, some gray areas, and some that are wrong. Do

not deviate dramatically from standard practice. If you choose to deviate substantially, take

more time to explain and get your method reviewed or even published. Besides, it is

5
professionally embarrassing when it’s revealed you caved to your boss; your reputation is a

repeat game.

You get the drift. Below is an abbreviated list of additional ways to lie and how to avoid lying.

HOW TO LIE HOW NOT TO LIE

1. Slant the question Neutrally identify the problem and its causation
This is all about greed; or jobs.
2. Assess only one alternative Assess reasonable alternatives including different
Rebuild the New Orleans levee to the sizes, different approaches, and doing nothing
height it was before
3. Ignore time “Discount” future values (the challenge is the
Just add up the new government appropriate rate).
revenue from gambling for the next 20
years.
4. Be vague about whose benefits and Clearly define who has standing (whose benefits
costs count and costs count). All citizens? Only those in a
Assess the impact of military base region?
closures only on the local region or be
unclear if the benefits of thievery accrue
to the thief.
5. Omit a summary table with Include a limited number of summary tables,
performance measures perhaps one for impacts in their natural units (e.g.
Make it hard to integrate the analysis, injuries, crimes), and one for the monetized
as with an analysis of the cost of values in each category.
sprinklers in nursing homes in which
benefits and costs were kept in different
units and distinct from each other.
6. Use misleading graphics or statistics See How to Lie with Statistics by Darrell Huff
For instance, truncate the vertical axis
so that what looks like a large change is
a small change compared to the total.
7. Ignore relevant differences among Take into account policy relevant differences such
people as those potentially associated with income,
In the benefits of emergency planning gender, age…
(or the cost of a disaster), assume an
average of 1.8 cars per household no
matter their income so that everyone
can drive out of harm’s way.
8. Ignore qualitative elements (not Qualitatively describe elements which cannot be
quantified or not monetized) quantified and put into monetary values.
Measure the cost of rape as hospital
and police reporting costs.

9. Never look back Plan for accountability by carrying out

Times have changed and it’s too hard to retrospective analysis.

6
 HOW TO LIE HOW NOT TO LIE

get information, let’s not see what

happened compared to what we
predicted!
10. And the first 6 in the text!

Of course, lying is a creative art and the salient methods listed above have infinite variation. No

group has a monopoly on the art; in my experience non-governmental organizations are as likely

as industry or government to shade the truth. At the same time, there are dedicated people in

each type of organization who are concerned about truth-telling both within their organization

and as watch-dogs across organizations. More power to them.