The Future of

Applied AI in
Ecommerce
The Future of Applied AI in Ecommerce

Commerce Has a Data Problem

With data coming from many different sources and We will focus on using AI for the four following
formats, brands and the vendors that support them objectives:
are struggling to connect the dots. They fail to
Develop your commerce intelligence to better understand
create the personalized shopping experience that 1 your customer and their product and content needs
consumers are starving for. AI is the best way to
take advantage of that data and apply it to produce Detect intent so you identify and unify all of the available data
2
a customized buyer’s journey. to organize information and support actual business outcomes

Applied AI leverages a set of technologies that augment human tasks Enrich your data and leverage existing source systems to
and solve business problems more accurately and efficiently. Applied AI 3 drive more acquisition, conversion, and revenue
typically encompasses machine learning, natural language processing,
and data from signals augmented with algorithmic intelligence.
Create a fully personalized and connected experience for
4 both your customer and your employee journeys with next gen
Ecommerce has been one of the most elegant industries to employ cognitive commerce only possible with AI
applied AI and realize immediate benefits. Many retailers are likely
deploying some form of AI-based technology in their ecommerce today.
In this white paper we’ll present a logical methodology for deploying AI
in stages across ecommerce operations and beyond. Enormous benefits
can be realized regardless of your vertical or category (B2B, B2C, DTC,
omnichannel, etc.),

© 2021 Lucidworks 2
 AI Transformed the Modern
Commerce Experience
Companies leveraging AI are realizing, on average, a 10-12% revenue gain1 and the
technology is expected to boost profitability by 59% by 2035.

AI PRIMARY BENEFITS TO COMPANIES

Enhance features, functions and/or performance of products 19% 10% 10% 51%

Make better decisions 14% 10% 10% 35%

Create new products 12% 12% 8% 32%

Optimize internal business operations 12% 14% 10% 36%

Free up workers to be more creative by automating tasks 10% 10% 16% 36%

Pursue new markets 8% 9% 8% 25%

Capture and apply scarce knowledge where needed 8% 9% 8% 25%

Optimize external processes like marketing and sales 7% 11% 12% 30%

Reduce headcount through automation 7% 8% 8% 23%

Don’t know 2% 2%

Rank 1st Rank 2nd Rank 3rd

© 2021 Lucidworks 3
 1
The Future of Applied AI in Ecommerce

Develop Your
Commerce IQ
Commerce intelligence requires continuous effort to learn and understand your shoppers, their
journeys, how they use your products, and how they interact with your brand. While this capability
is fundamental to any business, it’s crucial in ecommerce. However, only 14% of executives report
that they currently have truly customer-centric strategies. While product-centric strategies are
still important, combining the two is something AI can definitely enable.

COMMERCE
Existing Customers: Unkown Customers:
Profiles & History Referring / Landing Page

INTELLIGENCE
Visit Contexts Visit Contexts
Click Streams - Searches - Filters - Chats Click Streams - Searches - Chats
Posts & Reviews Purchases & Returns
Purchases & Returns Post Sale Interactions
Post Sale Interactions

AI
Touchpoint (ads, social, web, apps, Products & SKU:
call center, email, store, print) Use Cases
Optimal information and product Attributes & Taxonomy
feature set by context and touchpoint

91% of consumers are Optimal converting and non-

converting journeys by touchpoint
Content (user, customer or employee
generated)
Knowledge Base
more likely to shop with Cross touchpoint attribution and
influence by touchpoint 3rd Party Sources & Competitive

brands that recognize them Use case and product attributes by Supply Chain
touchpoints KPI’s

and provide them with a Multi-touch diagnosis

relevant experience.2
Touchpoint & Channel Intelligence Customer IQ Product IQ: Employee Interactions
Context Intelligence Target behaviors, characteristics, Top purchasing contexts and use cases (with customer, product, or
and profile customer feature and touchpoint):
Product Attribute Level Intelligence Top product features driving conversion
information affinities by context Marketing
Customer Dimension and Behavioral Behaviors driving purchases and returns
Intelligence Customer propensity by context Merchandising / Product
Target customer by context
Business Process Intelligence Customers conversion behaviors & Sales
journeys Attributes driving KPI’s
Support
Expert influencers or advocates Optimal taxonomy by context
Call Center / POS
Repurchase and/or loyalty Process Paths & Journeys

© 2021 Lucidworks 4
The Future of Applied AI in Ecommerce

AI can identify types of products and all of their respective attributes, in data from call centers and brick and mortar stores will also add
shopper personas, and how those products and branded content are an additional and vital intelligence layer. Augmenting all of this with
used throughout the buying journey. available demographic, psychographic, geographic, and knowledge
graph data provides additional training for the AI to capitalize on. The
This is critical to driving affinity towards your brand. AI can also uncover AI can then be used to validate and modify existing segmentation or
the relationships these insights have to your KPIs to help with problems customer intelligence you may already have in place.
such as cold start, private browsing, and cross-device mapping. This is
particularly important as privacy and data sharing constraints such as
AI really leaps ahead when going beyond segments
ITP, GDPR, and CCPA continue to expand.
and clusters, down to the individual shopper in the
moment.
Developing commerce intelligence can be done by instrumenting your
customer and employee-facing applications. AI can detect patterns
in employee/customer interactions and extract use cases from the Shopper behavior changes as the purpose of their site visit changes.
interaction content. Models can be trained on various information For example, during the holidays, when customers are not shopping for
sources surrounding your products such as product catalogs, attribute themselves and instead shopping for their family, friends, and even pets,
categories and seasonal marketing data. The context of your shopper AI can detect and understand these variations automatically.
interactions—the when, where, and how they purchase—will help
further elevate this capability. Thanks to event tracking and big data analytics, it is possible to quickly
uncover how your users shop, what is important to them, what drives
Just as important as why customers do buy, them to purchase, and more about their overall persona. While privacy
and compliance are always a concern, research shows that 91% of
is understanding why they don’t buy or leave
consumers are more likely to shop with brands that recognize them and
your site. provide them with a relevant experience.

This requires the collecting of active behavioral data –clickstream,

This brings us to Step 2, AI-powered needs and goal detection.
search, browse, email, and chat—and pairing it with information in
your analytics and customer data platforms (your CDP and/or CRM).
It’s also an essential step to be diligent about mining through historical
enterprise-wide data and connecting all of the dots there. Folding

© 2021 Lucidworks 5
2
The Future of Applied AI in Ecommerce

Detect Shopper
Needs, Goals,
and Intent
AI can infer shopper needs and their ultimate goals by mapping their interactions. Important
outcomes include detecting what product and content attributes are most important and
knowing which specific conditions certain products lead your shoppers to convert.

If your applications aren’t delivering the results or product attributes (color, model, size, etc.)
important to your users’ end goals, it not only impacts on-site conversions but also deeply
impacts acquisition such as SEO, affiliate and social media traffic. It is essential to understand
each shopper’s ultimate goal and intent so that you can deliver the right information they need
to make a purchase decision. This is both vital to driving conversions as well as overall business
optimizations (i.e., refining marketing campaigns, product planning, inventory planning, etc.).

83% of customer
experience stakeholders
cited linked business
outcomes to customer needs.

© 2021 Lucidworks 6
 A SUMMARY OF SHOPPER GOALS AND MOTIVATIONS IS DEPICTED
IN THE FOLLOWING DIAGRAM:

Activity Based Event Based Social Based B2B Based

A meal A job Seasonal Events Transitional Trending Reviews Based Team / Job
Events (ex. fashion / Department Requirements
A sport A project Holidays Gifting
regional) Needs
Entertainment Project
An art project A repair / Life Events Registries / Lists
Events Cohorts Application Requirements
upgrade
A trip Sport Event Guilt Requirements
Corporate Events Influencers Replenishment
Experts

Group Based Self Based Context Based

For family For the pets Health need Getting a Deal Security / Fear Regional / Local Conditions
members (other family) (ex. remedy) Trend (ex. security risk)
Emotionally / Comfort
For the home For the garage Beauty / Cosmetic Psychologically Seasonal Convenience
(ex. kitchen) (vehicles / tools) Greed
Age Related An investment Climate
Brand Trust
Fitness Save Time
Sense of Urgency

© 2021 Lucidworks 7
The Future of Applied AI in Ecommerce

In parallel to detecting intent and goals, AI can be

used to uncover important product attributes and
features.

By pairing shopper behaviors with underlying product attributes,

content, and corresponding interactions, you will uncover the attributes
and features that matter to customers.

For instance, a customer of yours might be looking for a specific brand

of vacuum cleaner and a certain model. After they find that vacuum
cleaner, they might start searching for other cleaning products and
home goods. Next time they visit your site, you can have those models
trained, through that valuable usage data, to show related home care
products. The system can also hide products they’ve purchased in the
past. So instead of showing other vacuum cleaners, the AI will show
filters and other parts of the previously purchased vacuum that might
need replacing over a certain period of time.

A common input to your machine learning application is your brand’s

interaction analytics across the web, call center, and brick and mortar
stores. Touchpoints like clickstream, searches, filtering, sorting, can
be coupled with purchases and cart abandonments. These will all give
you a good idea of what attributes and features should be prioritized to
improve ROI. This exercise can also be used to identify areas of sparse
data, and ideally, where to employ AI to fill in the blanks.

And this takes us to Step 3.

© 2021 Lucidworks 8
3
The Future of Applied AI in Ecommerce

Enrich Your Data

According to Dun & Bradstreet’s study, The Past, Present, and Future of Data, 25% of medium-
sized businesses have lost a customer due to incomplete or inaccurate information3. AI can
optimize your experiences and processes and enrich your product catalog and content through
deep learning, crawling, semantic extraction and signals.

But the number one hindrance to successfully bringing AI into your ecommerce organization will
be the lack of data and the quality of data that is available. Fortunately AI is a powerful tool for
improving the quantity and quality of data going into your systems of record.

The number of integrations and data available in and out of ecommerce systems has increased
exponentially. Most pronounced is the sheer volume of user-generated data which can include
analytics and user-generated content (reviews, FAQs, etc.). In addition, the volatility, velocity, and
complexity of the data available has also increased.

Lack of data can create the following performance challenges in

ecommerce:
• Customer acquisition impact such as not • Merchandising and personalization limitations
finding your products, brand, or content on because of lack of shopper behavior data

25% of medium-sized the web

• Low customer satisfaction caused by lack of
businesses have lost a • Product discovery challenges including not data, where shoppers can’t find what they want
customer due to incomplete or finding your products within your own site on-site or in the store
inaccurate information. through search and browse
• Less visibility & insight due to not being able
• Conversion problems from not having enough to connect your data across silos within your
information about a product for the shopper to organization
make a decision

• Lower lift such as in missing out on additional

revenue through better recommendations

© 2021 Lucidworks 9
 ECOMMERCE SOURCE DATA CHALLENGES*

AI can be used to identify data that should be included in the experience. Not all customers may be looking for products.

1 Sourcing Data
They might be looking for content, videos, testimonials, user generated content, reviews, offers and more. The same
entity extraction used to predict what your users are looking for paired with behavioral signals can be used to get a
clearer understanding of your customers’ information needs.

AI can be used to identify relationships between data within a single or multiple sources. This includes automatically
2 Linking associating content with products, uncovering contexts and behaviors without formal linking to content and products.
Deep learning takes this further such as using computer vision to detect relationships across images and content.

Sparse data is a well known issue in ecommerce. AI can be used to supplement data through encoding of signals,
semantics, NLP, sentiment, and context. User behavioral tracking, crawling, deep learning methods such as feature
3 Enriching extraction from text and images can also be employed. The enriched data can be placed in dense vectors to drive vastly
improved relevance, recall, precision, personalization and corresponding KPI’s.

Classification is done through human intuition, expertise based on theory or hypothesis on how users think about
products or content. AI can be used to auto - classify, label, and tag products through techniques such as feature
4 Classifying extraction, sentiment analysis, and supervised learning through contextual and behavioral signals. This will not only
automate classification but align it to how your users actually shop.

AI can be used to identifying data that should be included in the experience. Not all customers may be looking for

5 Cleaning
products. They might be looking for content, videos, testimonials, user generated content, reviews, offers and more. The
same entity extraction used to predict what your users are looking for paired with behavioral signals can be used to get
a clearer understanding of your customers’ information needs.

*Data processes can vary

© 2021 Lucidworks 10
 TYPICAL DATA SOURCES AND USES

Smart Product Merchandising Marketing Service Enterprise

Content Discovery

CMS/DAM, PIM/MDM, Automations, Curation, Analytics, Segments, Knowledge Base, Order Management,
Secured/Restricted, Product Content, Rules, Personalization, Audiences, Campaigns, FAQ, Chat, Order / Transactions, Planning/
User Generated/ Inventory, Pricing Recommendations Targeting, Journeys Shipment / Payment Forecasts, Financials,
Community Search Logistics

• Chatbot • Notifications
• Spec Sheets • Multi-Language • Spec Sheets • SEO
• Messenger • Triggers
• Safety Materials • Multi-Currency • Safety Materials • Email
• Self Service • Proximity
• Papers • Multi-Pricing • Papers • Mobile
• Call Center • Replenishment
• Guides • Multi-Discount • Guides • Integrated Ads
• Commerce • Margin / Cost
• Videos • Multi-Availability • Videos • Landing Pages
Integration
• Inventory Ratios
• UGC • Multi-Lead Time • UGC • Connected Exp.
• Cost/Margin
• Digital Assets • Digital Assets

© 2021 Lucidworks 11
The Future of Applied AI in Ecommerce

Now, how do we enhance the data, coming from all of these sources, in all of these formats? It’s no small feat.
Here’s how to ensure you’re making applied AI truly hum:

As you crawl all these multiple sources of information, and automatically identify important labels or tags to augment existing records you
1 have. As an example, crawling content pages, user generated content, and third sources which can add valuable information and context
surrounding your products, content, and brand.

AI can be used to link data by tagging and mapping relationships within algorithmic models. Lexical or semantic graphs can be created
2 so the machine can automate these relationships, thus organizing data in a way that actionable and can be surfaced by business users to
optimize in real time.

Use AI to enrich these massive data sets at scale by:

3
• Classification and tagging of your existing data to be aligned with the way your users think about products
• Identifying relationships between your records
• Eliminating noise by automatically deleting duplicate and garbage data
• Enriching through deep learning
• Leveraging signals to drive continual self-learning and ongoing optimization

Now, let’s talk about Step 4, where you will bring it all together to deliver a much more enhanced ecommerce experience.

© 2021 Lucidworks 12
4
The Future of Applied AI in ecommerce

Power AI-led
Experiences
AI-led experiences, sometimes called cognitive commerce, occur at the most mature stage in
the ecommerce journey. Here, AI is used to enhance the experience at every touchpoint, from
recommendations and personalization to merchandising and testing.

Wildly valuable tactics under this umbrella can also include:

• Semantic search
• Dense vectors
• Deep learning-based search

The reasons why to implement these types of applied AI methodologies are easy to triangulate.
Research shows that most shoppers leave a commerce site within 10 seconds of not finding what
they were looking for. Much of this can be attributed to product and content not being organized
in the way users think or shop.

10 seconds – how long In a perfect world, each experience would be tailored to the consumer in a hyper-personalized
before a shopper leaves a way. Products surfaced would feel one-to-one and companies could still prioritize and brand
site if they can’t find what their own initiatives within these deeply personal experiences. These tactics are the real-world
they are looking for. methodologies that allow us to do just that.

© 2021 Lucidworks 13
 TOOLS TO IMPROVE QUALITY OF RESULTS

Natural Language ML/AI Semantic Vectors

Processing

• Knowledge Base / • Clustering & Classification • Embeddings on

Dictionaries Documents & Queries
• Semantic Parsing
• Detection • Documents + Queries
• Signals / Predicted Intent
(ex. Synonyms, Speech)
• Documents + Semantics
• Head/Tail Analysis
• Analyzers
• Documents + Knowledge Graph
• Knowledge Graphs
• Token Filtering
• Documents + Context
• Behavioral Graphs
• Entity Extraction
• Documents + Behaviors
• Affinity / Propensity Models
• Documents + Affinity Models

© 2021 Lucidworks 14
The Future of Applied AI in Ecommerce

To illustrate how these tools work to enhance the

ELEMENTS OF PERSONALIZATION
journey, let’s delve into the most impactful AI-led
experience: personalization.

Context
According to Gartner4, personalized experiences lead to, on average,
Industry, Trade, Region, Season, Channel, Referrer, Landing
up to a 16% lift of impact of KPIs. AI can drive sympathetic experiences Page, Audience, Segment
across many or all of the elements of personalization. In the current
age of privacy, you can expect a significant amount of your traffic to
Profile
be anonymous and untraceable, therefore AI offers the most viable
Organization, Contract, Preferences, Omni-Channel Order
method to improve the performance of that traffic. History, Visit History, Affinities, Propensities, LTV

Through machine learning, personalization elements such as context Behaviors

and behaviors are associated with underlying signals and KPIs. These Keywords, Filters, Sorts, Click Through, Browse, Page Views
(Click Stream), Abandons, Carts, Orders
associations drive continuous improvement - the more data you have
about the user, the better the shopping experience you can deliver. All
of this can be married to work in harmony. Semantic
NLP/NLU, Ontology, Industry/Domain Dictionaries and
Knowledge Graphs, Synonym Learning
Finally, it’s essential to constantly measure and optimize. Your analytics
should link data across your entire organization. And with connected,
enriched data, applied AI can enable your brand to unlock customer Signals
Implicit, Explicit, System Search & Clickstream Meta-Data, Crowd,
insights, power predictions, automate optimization, and ultimately take
Transactional, Enterprise (inventory, margin, fulfillment), Sourced
your ecommerce experience to the next frontier.

Rules & Enrichment

Testing, Merchandising, Pin/Boost/Bury, Query Rewriting,
Redirects, Curated Experiences, Product & Content Enrichment

© 2021 Lucidworks 15
Bring Your Ecommerce
to a New Age of
Connected Experiences
Implementing applied AI can seem daunting, but once you’ve
developed your commerce intelligence, you can start to better
unify your data, understand your shoppers, and bring your
ecommerce to a new age of personalized, connected experiences. Get Started
This is where Lucidworks can help. We bring an AI-driven
ecommerce platform that powers some of the world’s largest Learn how Lucidworks can power connected
brands. Lucidworks Fusion combines domain knowledge with shopping experiences for your brand and meet
machine learning and AI-powered search capabilities to offer every customer’s need in the moment.
real-time hyper-personalized digital shopping experiences,
simplify shopper engagement and purchase paths, and increase
customer loyalty.

1
https://www.ibm.com/thought-leadership/institute-business-value/report/ai-value-pandemic
2
https://www.accenture.com/_acnmedia/pdf-83/accenture-pulse-check-infographic.pdf#zoom=50
3
https://www.dnb.com/perspectives/master-data/data-management-report.html
4
https://www.gartner.com/en/executive-guidance/personalization © 2021 Lucidworks