UCLA Big Time Research
UCLA Big Time Research
UCLA Big Time Research
Pauwels
Keywords: accountability, marketing effectiveness, efficiency, return on marketing investment, marketing value
assessment
The Difficulty of Marketing Value marketing investment (ROMI), remains an elusive goal for
Assessment most companies, which are struggling to integrate big and
small data and marketing analytics into their marketing
I want marketing to be viewed as a profit center, not a cost decision and operations.
center. Why is marketing value assessment so challenging? To
—A chief executive officer begin with, the term “marketing” refers to several things: a
I have more data than ever, less staff than ever, and more management philosophy (customer centricity), an organiza-
pressure to demonstrate marketing impact than ever. tional function (the marketing department), and a set of specific
—A chief marketing officer activities or programs (the marketing mix). However, regardless
arketing is at a crossroads. Managers are frustrated by of the intended use of the term, marketing aims to create and
when making resource allocation decisions. For example, and decisions exist in silos within the organization. However,
advertising is only deemed financially successful if its ability the consumer or customer is the target and recipient of all these
to increase awareness results in higher sales and/or profit actions, the combination of which will create the consumer’s
margins. attitude toward the brand and, eventually, his or her purchase
Current efforts in marketing measurement often do not go all behavior. In assessing marketing’s value, we therefore pay close
the way in connecting metrics to each other. For instance, many attention to the integration of marketing activities as they affect
balanced scoreboards and dashboards do not tell managers how consumer behavior. In this context, Court et al. (2009) argue that
their marketing inputs relate to customer insight metrics and to the critical task is to describe the process that generates sales for
product market performance metrics. Consistent with this notion, the firm and to identify the bottlenecks that impede profitable
in a personal communication, Lehmann uses the term “flow- business growth.
boards” for dashboards connecting metrics, while Pauwels (2014) In addition to relating performance metrics to each other (the
defines analytic dashboards as a concise set of interconnected metrics challenge), these metrics also need to be connected to
metrics. Indeed, reconciling multiple perspectives on marketing marketing activity. Indeed, assessing marketing value requires
value requires causality to be shown among marketing actions various demand functions that quantify how changes in mar-
and multiple performance outcomes (e.g., customer attitudes, keting activity influence changes in these dependent variables
product markets, financial markets; i.e., quantifying the arrows in (e.g., with response elasticities). Demand functions are often too
Figure 1). Connecting the metrics is especially challenging if data complex for senior managers to intuitively understand and
Customer • Causally close (often closest) to • Primary data may be difficult and costly • Sampling: current customers versus
mindset marketing actions to collect if direct from customers past customers versus all potential
• May be unique to marketing • Secondary data from research vendors customers in the marketplace
performance outcomes vs. other may not align well with theorized • Possible demographic effects on
business disciplines constructs or data from other vendors measures
• Commonly used to set marketing- • Noise in survey measures (primary
specific goals and assess marketing and secondary data)
performance in practice • Only allows for goal-based assessment
if collected with or supplemented by
primary data
• Transaction-specific versus overall
evaluations
Customer • Causally close to marketing actions • Primary data may be difficult and costly • Noise in survey measures (primary
behaviors • May be unique to marketing to collect if direct self-reports from data)
performance outcomes versus other customers • Only allows for goal-based assessment
business disciplines • Observed behavior data may require if collected or supplemented by primary
• Commonly used to set marketing- working with firms and can be difficult to data
specific goals and assess collect from multiple firms
performance in practice • Differences across firms in how
• Direct observation shows revealed observed behaviors are defined and
preferences calibrated
Customer- • Causally close to marketing actions • May require working directly with firms • Only allows for goal-based assessment
level • May be unique to marketing and may be difficult to work with multiple if collected or supplemented by primary
outcomes performance outcomes versus other firms data
business disciplines • Differences across firms in how • Noise in survey measures (primary
• Commonly used to set marketing- economic outcomes are determined data)
specific goals and assess and calculated
performance in practice
Product- • Causally close to marketing actions • Unit sales data are difficult to obtain • How to define the “market”
market- • May be unique to marketing from secondary sources for most • Only allows for goal-based assessment
level performance outcomes versus other industries if collected or supplemented by primary
outcomes business disciplines • Even firms in the same industry may data
• Commonly used to set marketing- differently define the markets in which • Noise in survey measures (primary
specific goals and assess they compete data)
performance in practice • Higher level of aggregation, so may be
less diagnostic
Accounting • Well-defined and standardized • Corporate level, so may be further away • Potential differences between firms and
measures from marketing actions and less industries in their accounting practices,
• Revenue-related items commonly diagnostic policies, and norms
used to set marketing-specific goals • Not forward looking • Differences in measures across
and assess marketing performance • May undervalue intangible assets countries
in practice • Mostly ignores risk • Only allows for goal-based assessment
• Secondary data availability • Treats most marketing expenditures as if collected or supplemented by primary
• For primary survey data, specific an expense data
items likely to have the same • Noise in survey measures (primary
meaning across firms data)
Financial • Investors (and analysts) are forward • Corporate level, so may be further away • Risk adjustment
market looking from marketing actions and less • Public/larger firm sample-selection bias
• May better value intangible assets diagnostic • Assumes primacy of shareholders
• Finance theory suggests that • Publicly traded firms only, which tend to among stakeholders, but this may not
investors may be more goal be larger be true in some countries
agnostic (but time frames and even • Difficulties in assessing firms across • Assumes the financial market is efficient
criteria may be goal related from the different countries (and financial and participants are well informed of the
firm’s perspective) markets) marketing phenomena being studied
• Secondary data availability • May be subject to short-term • Only allows for goal-based assessment
fluctuations unconnected with a firm’s if collected or supplemented with
underlying performance primary data
• Noise in survey measures (primary
data)
Resources Budget is received from senior management Budget is created for junior management
Objectives Efficiency, accountability of resource use Stimulating profitable growth for the brand or firm
Use of analytics Detailed analysis of (typically) one marketing-mix Integration across the marketing mix
element
Key challenges/risks Exaggerated belief in the strategic importance of Large financial consequences
one’s own silo
Examples Media-mix allocations Product portfolio decisions across international
Dynamic pricing markets
the method used, a critical question for management is whether In academic research, empirical generalizations on sales
market conditions will have changed by the time the actual response functions provide valuable guidance for marketing
decision is made. The beliefs that change outpaces analytic spending (Hanssens 2015). Table 3 provides a quantitative
insights and that past patterns do not apply to the future hinder overview, expressed as sales or market value elasticity esti-
the use of marketing analytics in many organizations. mates. These relate directly to marketing spending rules by
virtue of the fact that, at optimality, a firm should allocate re-
Findings on Marketing Investments and Allocations sources in proportion to its response elasticities (Dorfman and
Previously, we discussed investments and allocations in Steiner 1954). Table 3 also indicates the extent to which the
terms of their relationship to strategy and tactics. Next, marketing variable is an organic growth driver (i.e., its impact
we discuss findings more broadly. Table 2 shows dif- on sales is sustained rather than temporary). This is an im-
ferences between allocation and investment decisions on portant distinction because it identifies the strategic nature of
several fronts. Managers and academics are keenly interested marketing activities. Although price promotions and adver-
in decision rules for both, as is evident from the fact that this tising for existing brands (which often consume the majority of
topic appears frequently among the biennial research priori- marketing’s budget and effort) are not major organic growth
ties disseminated by the Marketing Science Institute. drivers of company performance, marketing assets (e.g., cus-
Notably, most applications in marketing analytics (includ- tomer satisfaction, brand equity) and actions (e.g., distribution,
ing analytics exploiting big data) focus on the deep dive for innovation) have a strong impact on long-term company value.
tactical allocations (see Table 2). Insofar as these contributions In an example from the French market, Ataman, Van Heerde,
overemphasize areas in which good data are readily available, and Mela (2010) demonstrated across 70 brands in 25 con-
they run the risk of being bogged down in details and failing sumer product categories that only breadth of distribution (.61)
to see the forest for the trees. In contrast, when complete and length of product line (1.29) had strong long-term sales
marketing-mix data are used along with econometric methods elasticities. By contrast, long-term elasticities of advertising
for inferring long-term impact, marketing analytics can also be (.12) and sales promotion (–.04) were small or negative.
very helpful for strategic investment decisions and for quan- At this point, generalizations—expressed as response
tifying risk in such decisions (e.g., Leeflang et al. 2009). elasticities—exist for many quantifiable marketing inputs,
TABLE 3
Response Elasticities Summaries
Typical Organic Growth
Elasticity Range Drivers (1) Driver?
Fourth, a dashboard may be used to communicate to language as influential executives. Marketing analytics
important stakeholders. The dashboard communicates not customers strongly differ in their decision-making lan-
only performance but also, through the choice of metrics, the guage, with some companies favoring a more analytic style
things an organization values. Vanguard’s dashboard, for ex- and others using a more intuitive style. We recommend
ample, enabled it to share with its corporate board its focus communicating marketing analytics according to the com-
on customer loyalty, feedback, and word of mouth. pany’s style.
Finally, a dashboard offers a good starting point for im- When decision makers have a more analytical style,
portant discussions, such as when management sets stretch presenting estimates and elasticities straight from the ana-
targets without providing additional resources. For instance, lytics helps them understand exactly what is going on and
the U.S. division of an automotive company was instructed how decision optimality is affected—for example, when
to increase profits despite longer innovation cycles and lower deciding how to allocate marketing budgets by drawing on
advertising budgets. Analytics and dashboard tools helped their relative elasticities. Even in such cases, though, it is best
the division present what-if scenarios and make its case to to provide the proper context—for example, by comparing
headquarters that trade-offs were necessary by quantifying the effects that television advertising elasticities and online
the relation between marketing actions and profits. advertising elasticities have on online performance metrics,
Dashboards also allow for more effective communication as Figure 4 shows.
with marketing partners, especially as companies move to Decision makers with a more analytical style require more
performance-based compensation of agency work. As the information on the analytics assumptions and the uncertainty
sales impact of performance metrics may differ across countries, around the performance projections. Academic researchers are
managers should use dashboard insights to set specific typically well versed in such explanations. In contrast, decision
metric targets (Pauwels, Erguncu, and Yildirim 2013). In makers with a more intuitive decision style may be averse to
the case of the U.S. division of the aforementioned auto- discussions on confidence intervals, functional form, and error
motive company, brand consideration was a more important distribution assumptions. Communicating analytics insights
performance metric in an emerging market, while brand liking in such environments requires more visualization, such as the
was more important in a mature market. Further research is heat map of the projected profit consequences of changes to
needed to generate empirical generalizations and boundary marketing actions shown in Figure 5.
conditions in this regard. Figure 5 shows the highest profit (8.51; units disguised)
as a specific combination of price ($45) and advertising
budget ($3.25 million) but also communicates how close
Adapting Communication to the Style of the other combinations are to this maximum projected profit. For
Decision Maker instance, at a current price level of $35, the decision maker
In their review of ISMS-MSI Practice Prize finalists, Lilien, may feel uncomfortable with prices over $40, perhaps fear-
Roberts, and Shankar (2013) detail the characteristics of success- ing a customer backlash not included as a model variable.
ful marketing science applications. They advocate estimat- The decision maker can look up the highest possible profit
ing simple, easy-to-use models and obtaining organizational and associated marketing actions for prices below $40. After
buy-in through, among other things, speaking the same adjusting the price in this model-suggested direction, more
data and insights will then be available for recalibration profit of 7.79) but also a quantitative argument for why profits
of analytics and intuition. Alternatively, the decision maker can be increased (up to) 9% if the advertising budget is increased
might decide to allocate only the $2 million communica- toward its optimal level. As such, the heat map enables deci-
tion budget provided by his or her superior, the investor (see sion makers to tweak model-derived optimal allocations, which
Table 1). The heat map provides the decision maker with not provides a level of decision comfort. Decision comfort has been
only the best outcome under the given budget (a projected shown to be an important contributor to managers’ willingness
FIGURE 5
Profit Heat Map of the Interaction of Price and Advertising
REFERENCES
Ailawadi, Kusum, Donald R. Lehmann, and Scott A. Neslin (2001), Colley, Russell H. (1961), Defining Advertising Goals for Mea-
“Market Response to a Major Policy Change in the Marketing sured Advertising Results. New York: Association of National
Mix: Learning from Procter & Gamble’s Value Pricing Strat- Advertisers.
egy,” Journal of Marketing, 65 (January), 44–61. Corstjens, Marcel and Jeffrey Merrihue (2003), “Optimal Market-
——— and Harald van Heerde (2015), “Consumer-Based and ing,” Harvard Business Review, (October), 3–8.
Sales-Based Brand Equity: How Well Do They Align?” working Court, David, Dave Elzinga, Susan Mulder, and Ole Jørgen Vetvik
paper, Tuck School of Business, Dartmouth College [available at (2009), “The Consumer Decision Journey,” McKinsey & Company,
https://www.researchgate.net/publication/282602155_Consumer- (accessed July 20, 2016), [available at http://www.mckinsey.
Based_and_Sales-Based_Brand_Equity_How_Well_Do_They_ com/business-functions/marketing-and-sales/our-insights/the-
Align]. consumer-decision-journey].
Albers, Sonke (1998), “A Framework for Analysis of Sources of DeHaan, Evert, Thorsten Wiesel, and Koen Pauwels (2015), “The
Profit Contribution Variance Between Actual and Plan,” Inter- Effectiveness of Different Forms of Online Advertising for Pur-
national Journal of Research in Marketing, 15 (2), 109–22. chase Conversion in a Multiple-Channel Attribution Framework,”
Ariely, Dan (2010), “Why Businesses Don’t Experiment,” Harvard International Journal of Research in Marketing, (published elec-
Business Review, (April), [available at https://hbr.org/2010/04/ tronically December 17), [DOI: 10.1016/j.ijresmar.2015.12.001].
column-why-businesses-dont-experiment]. Dekimpe, Marnik G. and Dominique M. Hanssens (1999), “Sus-
Armstrong, Scott (2001), Principles of Forecasting: A Handbook tained Spending and Persistent Response: A New Look at Long-
for Researchers and Practitioners. Berlin: Kluwer Academic Term Marketing Profitability,” Journal of Marketing Research,
Publishers. 36 (November), 1–31.
Ataman, Berk, Koen Pauwels, Shuba Srinivasan, and Marc Vanhuele Demirci, Ceren, Koen Pauwels, Shuba Srinivasan, and Gokhan
(2016), “Advertising’s Long-Term Impact on Brand Price Elas- Yildirim (2014), “Conditions for Owned, Earned and Paid Media
ticity Across Brands and Categories,” working paper, [available at Impact and Synergy,” Report 14-101, Marketing Science Institute.
http://ssrn.com/abstract=2783096]. Dorfman, Robert and Peter O. Steiner (1954), “Optimal Advertising
———, Harald J. van Heerde, and Carl F. Mela (2010), “The Long- and Optimal Quality,” American Economic Review, 44 (5), 826–36.
Term Effect of Marketing Strategy on Brand Sales,” Journal of Dwoskin, Elizabeth (2014), “Trends to Watch in 2015: From
Marketing Research, 47 (October), 866–82. Algorithmic Accountability to the Uber of X,” The Wall Street
Biesdorf, Stefan, David Court, and Paul Willmott (2013), “Big Data: Journal, (December 8), [available at http://blogs.wsj.com/digits/
What’s Your Plan?” McKinsey Quarterly, (March), [available at 2014/12/08/trends-to-watch-in-2015-from-algorithmic-accountability-
http://www.mckinsey.com/business-functions/business-technology/ to-the-uber-of-x/].
our-insights/big-data-whats-your-plan]. Farris, Paul W., Dominique M. Hanssens, James D. Lenskold, and
Blattberg, Robert C. and Stephen J. Hoch (1990), “Database Models David J. Reibstein (2015), “Marketing Return on Investment:
and Managerial Intuition: 50% Model + 50% Manager,” Man- Seeking Clarity for Concept and Measurement,” Applied Mar-
agement Science, 36 (8), 887–99. keting Analytics, 1 (3), 267–82.
Bloom, Nicholas, Luis Garicano, Raffaella Sadun, and John Van Germann, Frank, Gary L. Lilien, and Arvind Rangaswamy (2013),
Reenen (2014), “The Distinct Effects of Information Technology “Performance Implications of Deploying Marketing Analytics,”
and Communication Technology on Firm Organization,” Man- International Journal of Research in Marketing, 30 (2), 114–28.
agement Science, 60 (12), 2859–85. Golder, Peter N. and Gerald J. Tellis (1997), “Will It Ever Fly:
Bronnenberg, Bart J., Vijay Mahajan, and Wilfried Vanhonacker (2000), Modeling the Takeoff of Really New Consumer Durables,” Mar-
“The Emergence of Market Structure in New Repeat-Purchase keting Science, 16 (3), 256–70.
Categories: A Dynamic Approach and an Empirical Application,” Gupta, Sunil and Valarie Zeithaml (2006), “Customer Metrics and
Journal of Marketing Research, 37 (February), 16–31. Their Impact on Financial Performance,” Marketing Science, 25 (6),
Bucklin, Randolph E., Donald R. Lehmann, and John D.C. Little 718–39.
(1998), “From Decision Support to Decision Automation: A Hanssens, Dominique M. (2014), “Econometric Models,” in The
2020 Vision,” Marketing Letters, 9 (3), 235–46. History of Marketing Science, Russell S. Winer and Scott A.
Chen, Ming-Jer (1996), “Competitor Analysis and Interfirm Rivalry: Neslin, eds. Singapore: World Scientific Publishing, 99–128.
Toward a Theoretical Integration,” Academy of Management ——— (2015), Empirical Generalizations About Marketing Impact,
Review, 21 (1), 100–34. 2nd ed. Cambridge, MA: Marketing Science Institute.
CMO Survey (2016), “TopLine Results,” (February), (accessed ———, Leonard J. Parsons, and Randall L. Schultz (2001), Market
February 28, 2016), [available at http://cmosurvey.org/files/ Response Models: Econometric and Time-Series Analysis, 2nd
2016/02/The_CMO_Survey-Topline_Report-Feb-2016.pdf]. ed. Boston: Kluwer Academic Publishers.