5/25/23, 3:46 PM Gartner Reprint

Licensed for Distribution

Critical Capabilities for Application Performance Monitoring and

Observability
Published 8 June 2022 - ID G00750993 - 33 min read

By Gregg Siegfried, Mrudula Bangera, and 1 more

APM and observability tools provide visibility into the digital experience that optimizes user engagement. Use cases now include security,
and OpenTelemetry portends a new type of standardization. With many tools available, I&O leaders must identify their critical capabilities.

Overview
Key Findings
A surge in online channel demand and delivery continues unabated, even as the pandemic appears to slow. Interest in application performance monitoring and
observability tools remains high, as does innovation in the market.

An increase in well-publicized and damaging security breaches has led some APM and observability vendors to add security capabilities, most often a type of runtime
application self-protection.

The CNCF-curated OpenTelemetry distributed tracing standard is nearing completion and is becoming widely supported. The third element of the standard, logs, is due
by the end of 2022, at which time the first version will be complete. Many vendors represented in this research are already shipping implementations of OpenTelemetry.

Recommendations
I&O leaders responsible for infrastructure, operations and cloud management should:

Choose an application performance monitoring and observability solution by including use cases and personas outside IT operations, such as DevOps, site reliability
engineering and security operations. This encourages collaboration, maximizes the value of the tool and broadens the target audience.

Maximize future flexibility by requiring support for OpenTelemetry and other standards in your APM and observability product selection process. Even if you are not
adopting OpenTelemetry today, it is becoming an increasingly common way to collect performance data from external services.

Align your requirements, use cases and relevance to appropriate personas and the best-fit tool options by using this research, combined with Gartner’s Cool Vendors
research, to examine the rapidly shifting APM market.

What You Need to Know

Application performance monitoring (APM) and observability software enables the observation and analysis of application health, performance and user experience (UX).
It is used widely in enterprises by IT operations, site reliability engineers (SRE), cloud and platform operations, application developers, and product owners.

The key functional requirements of APM and observability tools include:

Observation of an application’s complete transactional behavior

Automated discovery and mapping of application and infrastructure dependencies (including cloud services)

Monitoring of applications running on mobile devices and desktop browsers

Identification and analysis of application performance problems and their impact on business outcomes

Native integration capabilities with automation and service management tools, as well as native integration with public cloud providers — for example, Amazon Web
Services (AWS) CloudWatch, Azure Monitor and Google Cloud Operations

Analysis of business key performance indicators (KPIs) and user journeys

Accept — that is, to identify

The ability to perform interactive interrogation of multiple telemetry types (such as traces, metrics and logs) to detect “unknown unknowns”
We use cookies to improve your experience
underlying causes of unexpected events

Application security
We use cookies functionality,
to deliver the bestoften delivered
possible via a common
experience agent orTo
on our website. framework
learn more, visit our Privacy
Policy. By continuing to use this site, or closing this box, you consent to our use of cookies. Cookie Notice.

https://www.gartner.com/doc/reprints?id=1-2A8YYHDZ&ct=220608&st=sb 1/16
5/25/23, 3:46 PM Gartner Reprint
Gartner has established six critical capabilities in the context of six use cases that differentiate the most-popular tools in this market:

Application debugging and distributed profiling (ADDP)

Behavior analysis

Business analysis

IT service and infrastructure monitoring

Root cause analysis

Runtime application self-protection (RASP)

The six use cases are:

Application owner/line of business (LOB)

DevOps/AppDev

Digital experience monitoring (DEM)

IT operations

Security operations

Site reliability engineering (SRE)/platform operations

This research focuses on the competitiveness of APM software products with respect to key functional dimensions, rather than the overall strengths and weaknesses of
the vendors themselves. This analysis complements the Magic Quadrant for Application Performance Monitoring, which defines the market and highlights a broad set of
factors, including corporate viability, vision, marketing and the geographic focus of the vendors that offer these APM software products. We recommend that organizations
use this research in conjunction with the Magic Quadrant, inquiries with Gartner analysts and other Gartner research to define their requirements and select the solutions
that best match their needs.

Analysis
Critical Capabilities Use-Case Graphics

We use cookies to improve your experience

We use cookies to deliver the best possible experience on our website. To learn more, visit our Privacy
Policy. By continuing to use this site, or closing this box, you consent to our use of cookies. Cookie Notice.

https://www.gartner.com/doc/reprints?id=1-2A8YYHDZ&ct=220608&st=sb 2/16
5/25/23, 3:46 PM Gartner Reprint
Vendors’ Product Scores for IT Operations Use Case

Source: Gartner (June 2022)

Vendors’ Product Scores for Security Operations Use Case

We use cookies to improve your experience

We use cookies to deliver the best possible experience on our website. To learn more, visit our Privacy
Policy. By continuing to use this site, or closing this box, you consent to our use of cookies. Cookie Notice.

https://www.gartner.com/doc/reprints?id=1-2A8YYHDZ&ct=220608&st=sb 3/16
5/25/23, 3:46 PM Gartner Reprint

Source: Gartner (June 2022)

Vendors’ Product Scores for Application Owner/Line of Business Use Case

We use cookies to improve your experience

We use cookies to deliver the best possible experience on our website. To learn more, visit our Privacy
Policy. By continuing to use this site, or closing this box, you consent to our use of cookies. Cookie Notice.
Source: Gartner (June 2022)

https://www.gartner.com/doc/reprints?id=1-2A8YYHDZ&ct=220608&st=sb 4/16
5/25/23, 3:46 PM Gartner Reprint

Vendors’ Product Scores for DevOps/AppDev Use Case

Source: Gartner (June 2022)

Vendors’ Product Scores for Digital Experience Monitoring Use Case

We use cookies to improve your experience

We use cookies to deliver the best possible experience on our website. To learn more, visit our Privacy
Policy. By continuing to use this site, or closing this box, you consent to our use of cookies. Cookie Notice.

https://www.gartner.com/doc/reprints?id=1-2A8YYHDZ&ct=220608&st=sb 5/16
5/25/23, 3:46 PM Gartner Reprint

Source: Gartner (June 2022)

Vendors’ Product Scores for SRE/Platform Operations Use Case

We use cookies to improve your experience

We use cookies to deliver the best possible experience on our website. To learn more, visit our Privacy
Policy. By continuing to use this site, or closing this box, you consent to our use of cookies. Cookie Notice.
Source: Gartner (June 2022)

https://www.gartner.com/doc/reprints?id=1-2A8YYHDZ&ct=220608&st=sb 6/16
5/25/23, 3:46 PM Gartner Reprint

Vendors

Alibaba Cloud

Alibaba’s APM and observability offering evaluated in this research consists of three products: Application Real-Time Monitoring Service (ARMS) for application
monitoring, Cloud Monitor for resource and infrastructure monitoring, and Log Service for log file aggregation and analysis. The tools are optimized for running inside the
Alibaba Cloud ecosystem, with an option for on-premises deployment for large clients. Limited support is available for cloud providers other than Alibaba.

Alibaba’s offering scored above the mean on the majority of critical capabilities. Alibaba’s highest score was in RASP, using risk analysis components built into the ARMS
product. The lowest score was on behavior analysis, with fewer offerings designed for DevOps personas.

Alibaba’s suite has gained significant enhancements since last year, including integrated dashboards via Grafana, increased security functionality and improved business
analysis tools. It has strong integrations with other parts of the Alibaba suite, such as Enterprise Distributed Application Service (EDAS; Alibaba’s PaaS solution) and
DingTalk.

Alibaba Cloud monitoring is best suited to the application owners/line of business use case.

Amazon Web Services

The Amazon Web Services (AWS) APM and observability product evaluated in this research comprises Amazon CloudWatch, AWS X-Ray, Amazon CodeGuru and Amazon
DevOps Guru. It enables observability use cases and is not packaged or positioned as an APM solution. Recent releases of CloudWatch have expanded DEM with the
addition of CloudWatch Synthetics, CloudWatch RUM and CloudWatch Evidently.

AWS’s offering scored above the mean on the IT service/infrastructure monitoring capability. AWS also scored above the mean on the RASP critical capability due to
CloudWatch’s ability to block an attack-in-progress using a custom web application firewall (WAF) rule.

The AWS APM and observability solution is best suited for the IT operations and security operations use cases.

Broadcom

Broadcom’s products evaluated in this research are DX APM SaaS. Available for on-premises or as software as a service (SaaS), both options provide functionality and
code parity in the most recent release. The APM and observability products form part of Broadcom’s suite of monitoring, and are designed to work with its artificial
intelligence for IT operations (AIOps) products. Broadcom offers a wide array of enterprise software, including mainframe automation, testing, security and portfolio
management.

Broadcom’s APM offering scored above the mean on RCA, IT service monitoring, and application debugging and profiling. Broadcom’s solutions for IT service monitoring
are a key strength for this provider, achieving Broadcom’s highest score along with application debugging and distributed profiling. Broadcom’s security offerings, largely
from Symantec, are offered stand-alone from DX APM tools. Broadcom’s APM product scored its lowest in business analysis.

Broadcom’s APM offering is best suited for the IT operations and SRE/platform operations use cases.

Cisco (AppDynamics)

Cisco’s APM and observability suite evaluated in this research consists of AppDynamics, ThousandEyes, Intersight Workload Optimizer and Cisco Secure Application. It
provides full-stack observability, consisting of infrastructure performance, application performance, UX, business context and security capabilities.

Cisco (AppDynamics) scored above the mean on all critical capabilities. The offering continues to excel in business analysis and RASP. Cisco (AppDynamics)’s lowest
score is in application debugging and distributed profiling, because the product has historically served operations, rather than development teams.

Cisco (AppDynamics) has strong functionality for monitoring and analyzing complete user journeys, using AppDynamics Business iQ and Experience Journey Maps. This
helps operations teams prioritize responses based on business impact and UX. Cisco (AppDynamics) also has a strong security product, Cisco Secure Application. It can
provide operations and security teams with visibility into application vulnerabilities and threats for better joint risk assessment and enforcement, with alignment across
security operations and ITOps/DevOps teams. The analytics built on the platform generates dashboards for analysis of business performance with application
performance.

Cisco (AppDynamics) offering is best suited for the application owners/line of business and RASP use cases.

Datadog

Datadog’s platform is based on a unified interface with integrated components for each core functional area. For APM, the core modules include APM, Continuous Profiler,
Serverless Monitoring, Log Management, Database Monitoring and Real-User and Synthetic Monitoring. Other recent additions include components for security and
network monitoring.

Datadog’s APM offering scored above average on all critical capabilities, with Datadog frequently appearing near the top of most categories. The offering scores high in
application debugging, distributed tracing and continuous profiling. Datadog excels in distributed tracing with no sampling. Datadog’s lowest score was in the category of
We use cookies to improve your experience
RASP. After this research was completed, Datadog announced the launch of its application security product, which was not evaluated for this research.

We usehas
Datadog cookies to deliver
broadened the bestinpossible
its portfolio experience
recent years beyondoncore
our infrastructure
website. To learn more, visit
monitoring, andour
thisPrivacy
has been reflected in its growing customer base. The majority of
Policy.clients
Datadog By continuing
actively to
useuse this site,
multiple or closing
products fromthis
thebox, you consent
portfolio, to our
with most useinfrastructure
using of cookies. Cookie Notice.
monitoring, log management and synthetic monitoring alongside APM.

https://www.gartner.com/doc/reprints?id=1-2A8YYHDZ&ct=220608&st=sb 7/16
5/25/23, 3:46 PM Gartner Reprint
Datadog has strong functionality in correlating traces where it connects distributed traces to infrastructure, metrics, logs, network calls and processes in a single screen
for troubleshooting application performance.

This product is well-suited for midsize to large enterprises, particularly those with applications deployed in cloud or hybrid environments. Based on Gartner’s analysis, the
product is well-aligned with the behavior analysis use case.

Dynatrace

The Dynatrace platform was evaluated in this research. The product is offered as a SaaS or managed service, both based on the same code. The Dynatrace platform, in
addition to its core APM functionality, offers other capabilities via products such as IT infrastructure monitoring (ITIM), DEM, business analytics, AIOps, cloud automation
and application security.

Dynatrace scored well above the mean on all critical capabilities. The offering continues to excel in CloudOps, ADDP, behavior analysis and RCA. Its AIOps engine, Davis, is
embedded across the platform and examines data across modules and data types. Dynatrace’s PurePath enables users to detect performance problems and probable root
cause with relative ease, compared with other offerings. Dynatrace’s lowest score is in business analysis and RASP, as this newer module continues to evolve and mature.

Dynatrace has strong functionality to establish, monitor and evaluate service-level objectives (SLOs) and error budgets. Dynatrace also can receive flag changes and
display data in graphs to illustrate the impact of feature rollouts on performance. Its capabilities in detecting network or internet service provider (ISP) issues is more
basic, compared with other solutions.

Dynatrace’s offering is best suited for the IT operations and SRE/platform operations use cases.

Elastic

Elastic’s product evaluated in this research is Elastic Observability, Version 8.0. The product is available both as a SaaS offering and for on-premises deployment. The
broader Elastic portfolio includes products for enterprise search, security and the traditional Elastic Stack, aka ELK (Elastic, Logstash, Kibana).

Elastic scored above the mean on all capabilities, scoring highest on behavior analysis and business analysis. Elastic also scored well for real-time application self-
protection capability, with the malware protection and endpoint protection integrated in the Elastic agent.

Elastic products are built on top of open-source technologies. Elastic differentiates itself from its competitors with a simplified pricing model and the fact that the
underlying Elasticsearch architecture is highly flexible. This supports the ingestion of a wide variety of data types, which can be used as part of different analyses.

Elastic’s APM offering is best suited for the SRE/platform operations and DevOps/AppDev use cases.

Honeycomb

The APM and observability product evaluated in this research is Honeycomb. The Honeycomb service comprises a data store and query engine optimized for exploratory
identification and investigation of patterns and anomalies in application and infrastructure telemetry. It is not a traditional APM tool, and is most accurately characterized
as an observability platform. With its analytics engine and native support for OpenTelemetry, Honeycomb handles a variety of APM use cases.

Honeycomb scored above the mean on the RCA, IT service monitoring, ADDP and behavior analysis critical capabilities. It was strongest in behavior analysis, which
evaluates observability characteristics. The overall scores for Honeycomb were affected by the vendor’s failure to demonstrate capabilities in certain scenarios. The lack
of an integrated security option results in Honeycomb being marked “N/A” for that use case.

Honeycomb’s offering is best suited for the DevOps/AppDev and SRE/platform operations use cases.

IBM (Instana)

IBM’s APM and observability product is IBM Observability by Instana APM, and is largely derived from the December 2020 acquisition of Instana. IBM also offers an IBM Z
Mainframe version of the product, which is not covered in this research. Instana does not have native synthetic monitoring tools, but partners with a third-party product,
Apica. Instana does not have integrated capabilities for security, relying on other parts of the IBM portfolio, such as StackRox.

Instana scored slightly above the mean for most critical capabilities. Its best showing was in the area of application debugging and distributed profiling. The lack of an
integrated synthetic monitoring and security solution, in particular, limited its abilities for some critical capabilities.

Based on Gartner’s analysis, the product is best aligned with the IT operations and SRE/platform operations use cases.

Logz.io

The Logz.io APM and observability product set comprises distributed tracing, infrastructure monitoring and log management. Logz.io’s products are based almost entirely
on open source. For storage, the vendor leverages Prometheus-compatible M3DB for metrics and OpenSearch for logs and traces. OpenTelemetry is used to collect trace
data, which is visualized in Jaeger. Logz.io initially focused on log monitoring built on Elasticsearch and has expanded into broader monitoring and observability use cases.

Logz.io scored below the mean for all the critical capabilities. Its highest score was in RASP. The quality of open-source monitoring tools is good, and Logz.io has chosen
high-quality projects with effective communities on which to base its APM and observability offerings. Logz.io’s scores were affected by the large number of
We use cookies
demonstration to that
elements improve your
were not experience
attempted.

Logz.io’s
We useAPM and observability
cookies to deliver the product is bestexperience
best possible suited for the security
on our operations
website. To learnuse case.
more, visit our Privacy
Policy. By continuing to use this site, or closing this box, you consent to our use of cookies. Cookie Notice.
ManageEngine

https://www.gartner.com/doc/reprints?id=1-2A8YYHDZ&ct=220608&st=sb 8/16
5/25/23, 3:46 PM Gartner Reprint
ManageEngine’s APM offering consists of two separate tools: ManageEngine Applications Manager (Version 15.4), which is an on-premises offering, and Site24x7 for
SaaS deployments. ManageEngine is a subsidiary of Zoho Corp. Although the two share common code, they do not have feature parity. There are many other products in
the ManageEngine portfolio, including service desk, endpoint management and identity management tools.

ManageEngine scored below the mean for all critical capabilities. It scored highest for business analysis and IT service monitoring. Although ManageEngine scored lowest
on RASP, because it does not have an integrated security offering, there are stand-alone security information and event management (SIEM) and security auditing tools in
the wider portfolio.

Both products are best suited to small or midsize businesses (SMBs). Based on Gartner’s analysis, the product is best aligned with the IT operations use case.

Microsoft

Microsoft’s APM and observability product evaluated in this research is Microsoft Azure Monitor. It is part of the larger set of cloud-based products under Microsoft Azure
and is offered only as SaaS. The APM and observability functionality in Azure Monitor comprises Application Insights and Log Analytics.

Microsoft scored at or above the mean on all critical capabilities. Its strongest demonstrated capabilities were for business analysis and behavior analysis. Microsoft
Azure Monitor is tightly integrated into the provider’s development and operations tools, and possesses good capabilities for application debugging and distributed
profiling. The strong functionality and application maps are somewhat offset by workflows that require high knowledge of the product and environment being analyzed.

Microsoft’s APM and observability product provides tight integration with the vendor’s ecosystem, such as Azure DevOps and GitHub, but also extends bidirectional
integration with many other popular DevOps tools. Microsoft Azure Monitor lags behind in monitoring performance at endpoints and isolation of performance problems
caused by network traffic, which required a separate product.

Microsoft’s APM and observability offering is best suited for the application owner/line of business and DevOps/AppDev use cases.

New Relic

New Relic’s APM product evaluated in this research is part of the New Relic One platform. The platform is offered exclusively as a SaaS deployment. It can ingest telemetry
from multiple environments into its data platform.

New Relic scored above or on the mean for most critical capabilities. Its highest scores were in behavior analysis and application debugging and distributed profiling. It
also scored well in IT service monitoring. Its lowest scores were in RASP (where it has yet to bring a security offering to market) and in business analysis.

New Relic products have a particular focus on developers and SREs. The vendor recently expanded its portfolio to include Codestream, a plug-in that enables developers to
use their integrated development environments (IDEs) to debug production code. New Relic is also adding new telemetry types to the platform, such as extended Berkeley
Packet Filters (eBPF) and enhanced support for OpenTelemetry.

New Relic One is best suited for the SRE/platform operations and DevOps/AppDev use cases.

Oracle

Oracle’s APM and observability product evaluated in this research is the Oracle Cloud Observability and Management (O&M) platform, a SaaS solution that supports
database, infrastructure and APM in the OCI and Oracle enterprise applications customer base. Oracle positions O&M similarly to other cloud providers that offer
management platforms. The APM and observability capabilities support multicloud environments, but are optimized for the Oracle stack.

Oracle’s APM and observability offering scored above the mean on root cause analysis, near the mean on RASP and below the mean on the remaining critical capabilities.
Oracle’s highest scores were in root cause analysis and ADDP. Oracle’s lowest score was in business analysis.

Oracle’s O&M product offers visually appealing dashboards and interactive analysis of trace telemetry via the Trace Explorer. Use of the Apdex in the APM and
observability software is pervasive. The Apdex is present in multiple competitive products as well.

Oracle is best suited for the DevOps/AppDev and SRE/platform operations use cases.

Riverbed (Aternity)

Riverbed’s Aternity APM product evaluated in this research is Aternity Digital Experience Management, Version 2022.2. The Aternity platform comprises APM, End User
Experience Monitoring (EUEM), synthetic transaction monitoring (STM), digital product management (DPM) and IT service benchmarking capabilities. Aternity is now part
of the Riverbed portfolio again and continues to provide integration with products from the Riverbed monitoring family.

Aternity scored above or close to the mean on most critical capabilities. It scored highest on the business analysis use case and lowest on the RASP use case. Aternity
continues to excel at use cases involving DEM, which is one of the vendor’s primary focus points.

Aternity differentiates itself from competitors with a strong focus on DEM and transaction-level monitoring for end-to-end visibility for all types of applications, including
managed, desktop and SaaS. As with previous versions, Aternity is the only vendor evaluated in this research with a dedicated endpoint agent. Aternity also has a unique
benchmarking feature that enables clients to rapidly compare themselves with their peers, based on industry, geography and other categories.
We use cookies to improve your experience
Aternity’s offering is best suited for the application owners/line of business and digital experience monitoring use cases.

We use cookies to deliver the best possible experience on our website. To learn more, visit our Privacy
SolarWinds
Policy. By continuing to use this site, or closing this box, you consent to our use of cookies. Cookie Notice.

https://www.gartner.com/doc/reprints?id=1-2A8YYHDZ&ct=220608&st=sb 9/16
5/25/23, 3:46 PM Gartner Reprint
SolarWinds offers a SaaS-based suite named APM Integrated Experience that is an integration of its separate tools: AppOptics, Loggly and Pingdom. The SaaS suite is
optimized for cloud-native environments and supports code-level instrumentation and infrastructure monitoring. The product also supports on-premises and traditional
environments. SolarWinds’ Server & Application Monitor (SAM) product has a reduced APM feature set for on-premises monitoring.

SolarWinds scored below the mean for all of the critical capabilities, with its strongest performance in the IT service monitoring space. SolarWinds lowest score was in
RASP, because it does not have an integrated security offering. It also scored low on business analysis due to the limited displays available for non-IT users. This remains
consistent with the vendor’s primary customer being the IT professional in the organization.

SolarWinds is best suited for the IT operations and SRE/platform operations use cases.

Splunk
The Splunk APM and observability product evaluated in this research is the Splunk Observability Cloud, a SaaS solution that supports applications, infrastructure, logs and
DEM. The solution relies on OpenTelemetry and includes broad support for cloud-based infrastructure and applications built on cloud-native platforms, but self-hosted,
more-traditional workloads are supported as well. Although it is built largely as an amalgam of multiple acquisitions, the level of integration between the parts, as well as
with other rest of the Splunk portfolio, is much improved.

Splunk scored above the mean for all critical capabilities, except RASP. This represents a significant increase over prior years. Its highest scores were in the root cause
analysis and ADDP critical capabilities.

Splunk is best suited for the IT operations and SRE/platform operations use cases.

Sumo Logic

The Sumo Logic APM and observability product evaluated in this research is Sumo Logic Application Observability, a SaaS-only solution that supports application,
infrastructure, log and digital experience monitoring. The solution supports OpenTelemetry and includes broad support for modern applications built on cloud-native
platforms as well as self-hosted, traditional workloads. Sumo Logic analytics are built using both unsupervised machine learning (ML) and statistical techniques.

Sumo Logic scored above the mean for all critical capabilities. Its highest scores were in the RASP capability.

Sumo Logic is best suited for the RASP and IT operations use cases.

VMware (TO)

VMware’s APM and observability product Tanzu Observability (TO) by Wavefront was evaluated in this research. Tanzu Observability includes more observability than APM
features and has been used by enterprises, startups and service providers as a time-series-based monitoring platform. Wavefront includes support for OpenTelemetry
distributed tracing, which is how the product supports APM use cases.

VMware scored below the mean on all critical capabilities, with its strongest performance in application debugging and distributed profiling. TO’s lowest score was in
business analysis, because the product does not have native support for monitoring application user journeys. The lack of an integrated security option results in TO being
marked “N/A” for that use case.

VMware is best suited for the DevOps/AppDev and SRE/platform operations use cases.

Context
Given the increased adoption of hybrid and multicloud technologies to support digital transformation initiatives, APM and observability software solutions have become
critical to the success of a growing set of stakeholders in IT, software development and the business. These solutions help organizations improve UX, shorten mean time
to repair (MTTR), improve service availability, detect anomalous application behavior and forecast future problems. They also provide critical data that is useful outside a
purely IT operations context, including insight into the impact that performance has on customer behavior and business processes.

APM and observability software is part of an overall enterprise availability and performance monitoring portfolio, and its adoption becomes more important due to a
confluence of factors, including:

The growing trend and need for digital transformation driving the importance of the health of the application, which directly corresponds to the health of the business

Increasing application architecture and deployment complexity; on-premises, hybrid, cloud-native and multicloud deployments; and an uptake in the adoption of
containers, Kubernetes, microservices and serverless computing

The increasing velocity of application releases, configuration changes and integrations

The large and varied number of potential endpoints from which the consumption of enterprise business services occurs

Product/Service Class Definition

APMWeanduseobservability
cookies to software
improvesolutions provide for the collection, aggregation and analysis of data to assess application performance. These tools are increasingly
your experience
being tasked to solve difficult issues in diverse and complex application architectures — on-premises, in the cloud and hybrid — using the APM critical capabilities
identified in the Critical Capabilities Definition section of this research.
We use cookies to deliver the best possible experience on our website. To learn more, visit our Privacy
Policy. By continuing to use this site, or closing this box, you consent to our use of cookies. Cookie Notice.
Critical Capabilities Definition
https://www.gartner.com/doc/reprints?id=1-2A8YYHDZ&ct=220608&st=sb 10/16
5/25/23, 3:46 PM Gartner Reprint

Business Analysis

Business analysis identifies the health of the organization with respect to its KPIs, including financial metrics. Business processes describe and, using APM, track its flow
and status across the underlying infrastructure.

Business analysis goes beyond performance and availability statistics. It provides insight into LOBs, including for digital marketers and application developers, on the
efficacy of the services being offered. Shopping cart abandonment rates, shopping cart conversions and even the financial value of a transaction are some of the
measures to look for in this area.

Root Cause Analysis

Root cause analysis attempts to determine the probable cause of a problem, then construct a causality chain that links cause and effect. Much of the activity of APM
technology focuses on detecting and providing insight into root causes.

The probable cause (for example, a failure or degradation) is linked on a graph to the entities (e.g., application, services or infrastructure) that are affected or will be
affected. This methodology supports differentiation between cause and effect, and prioritizes repair on the cause, rather than attempting to fix transient symptoms.

ITService/Infrastructure Monitoring

This reflects whether agreed-on commitments regarding responsiveness and uptime are being met. This includes identifying adherence to contracted service-level
availability and performance thresholds.

Often spelled out in the form of service-level agreements (SLAs), operational-level agreements (OLAs) and SLOs, these measures provide visibility to line-of-business and
application owners, as well as to IT operations and DevOps teams, on the health of key services. The metrics provide a common view or boundary object that enables
communication across these disparate teams.

The inclusion of infrastructure monitoring here reflects the sensitivity that many organizations have regarding monitoring costs and agent sprawl. Because APM and
observability solutions often presume compute instances or VM-level agents for collecting telemetry from applications, collecting and analyzing, infrastructure-level
telemetry is increasingly an expectation.

ADDP

Application debugging and distributed profiling is the process of identifying the sources of defects and errors in computer and program code and mitigating performance
degradation.

Application debugging is a primary activity of application developers and, sometimes, application support teams, in the process of finding the causes of undesirable
program effects. Distributed profiling focuses on identifying and localizing the sources of performance degradation across an infrastructure made up of applications,
services and machines.

RASP
Runtime application self protection is a type of application shielding, located on a workload or client endpoint, that can identify vulnerable components at runtime and, in
some cases, block attempts to exploit them.

The RASP features of APM and observability solutions offer an additional layer in a defense-in-depth strategy, rather than removing the need for other security solutions.
This critical capability was introduced this year, so it is not unexpected that most vendors’ lowest scores were in this area.

Behavior Analysis

Behavior analysis facilitates the discovery and classification of user and application behavior by providing guided or ad hoc access to raw telemetry in a manner that
supports iterative, hypothesis-driven and exploratory analysis.

This capability highlights the maturity of the observability component in APM and observability solutions.

Use Cases

IT Operations

IT operations teams are usually tasked with keeping the infrastructure operating efficiently. However, its areas of responsibility can vary widely.

IT operations practitioners may be in specific silos, such as the network team, server team, virtualization team, application support team or other cross-functional teams
(such as a generalized monitoring team). Most use of APM and observability tools in traditional IT operations teams involves generalists tasked with identifying problems
quickly. This is accomplished by using service monitoring and anomaly detection for early-warning notifications of performance issues.

Security Operations

Security operations is tasked with maximizing organizational resilience against business risks, threats and exposure, and enabling prevention, detection and response.
We use cookies to improve your experience
Security operations is also responsible for the security operations center (SOC), which is the command, control and response element for security-related events and
incidents.
We use Security
cookies operations is a
to deliver the newpossible
best use case for APM and
experience observability
on our website. Toin learn
this edition of the
more, visit Critical
our Capabilities.
Privacy
Policy. By continuing to use this site, or closing this box, you consent to our use of cookies. Cookie Notice.

https://www.gartner.com/doc/reprints?id=1-2A8YYHDZ&ct=220608&st=sb 11/16
5/25/23, 3:46 PM Gartner Reprint
Application vulnerabilities are responsible for many of the high-profile breaches and intrusions that receive news coverage and are damaging to the reputation and health
of the affected organizations. The trace telemetry that APM and observability solutions collect to monitor performance includes valuable security signals as well. Although
today’s implementations are quite nascent, the security capabilities of APM and observability tools have proved to be valuable. The Log4Shell incident in late 2021, in
which a long-standing, but recently discovered, vulnerability was being widely and actively exploited, was an outstanding proving ground.

Digital Experience Monitoring

This is associated with multiple roles in organizations that use APM and observability tools to understand application performance and usability characteristics.

DEM technologies monitor availability, performance and experience quality as end users interact with applications and supporting infrastructure. Users can be external
consumers of a service (such as patrons of a retail website), internal employees accessing corporate tools (such as a benefits management system) or a combination of
both. DEM technologies look to observe and model the behavior of users as a continuous flow of interactions in the form of user journeys.

DevOps/AppDev

DevOps release teams must rapidly assess the quality (and effectiveness) of recent releases to detect and diagnose issues quickly.

This often requires in-depth data sourced from multiple, integrated tools. DevOps teams also help plan for future requirements and, therefore, look to assess the impact of
recently performed or planned application or infrastructure changes. Application developers increasingly make APM buying decisions, due to changes in how IT teams
support the deployment of emerging technologies with cloud-native architectures, as well as increasing the separation of responsibilities between core IT functions and
developers.

Ultimately, the role responsible for developing and supporting software visibility and instrumentation is a key element. This role also includes some elements of application
testing through the application life cycle. A key buying criterion of application developers is method-level code visibility, including the ability to compare and profile code in
production environments. Problem isolation is another key capability that is often shared with application support. Infrastructure visibility provides increased
understanding of the impact the code has on infrastructure elements.

DevOps and App/Dev use cases generally have the same APM and observability requirements, although they are often distinct functions in an organization.

Application Owner/Line of Business

Application owners, LOB managers and those responsible for service delivery from a nontechnical perspective are becoming interested in APM technologies.

These users formerly had been provided limited infrastructure metrics and health data from IT operations organizations. However, because this information was not
enough to make business decisions, LOB buyers are driving APM suite tool purchases. The primary focus of these users is on business metrics and customer journeys, but
end-user experience data is also important to detect trends before they manifest into something that might harm the business.

SRE/Platform Operations

This provides a self-service development, deployment and operational platform that enables multiple software delivery teams to build and operate their own products.

SRE enables cross-functional teams to design and operate scalable, resilient systems with complex modern architectures, usually in cloud environments.

As development of cloud-native applications continues to increase, APM and observability vendors have continued to evolve their support for applications built, using
microservices, packaged in containers and deployed across flexible, intelligent infrastructures. Typically, the containers are orchestrated using Kubernetes. As such, the
line between the application and the intelligent infrastructure becomes blurred. This is especially true as cloud-native applications are isolated or abstracted from server,
OS and even traditional network dependencies. These architectures are of enormous value to the business in terms of cost and agility, and they present new challenges to
monitoring tools, such as the short lifetime of containers, which is often only seconds.

Most often, platform operations or SRE are the personas managing cloud-native applications and responsible for their performance and availability. Their skills are a
combination of DevOps and traditional ITOps skills, and they work with cloud management, infrastructure as code (IaC), and monitoring and observability tools, such as
those from these vendors.

Vendors Added and Dropped

Added
Amazon Web Services

Honeycomb

Logz.io

SumoLogic

VMware (TO)

We use cookies to improve your experience

Dropped
Tingyun did not meet the inclusion criteria.
We use cookies to deliver the best possible experience on our website. To learn more, visit our Privacy
Inclusion Criteria
Policy. By continuing to use this site, or closing this box, you consent to our use of cookies. Cookie Notice.

https://www.gartner.com/doc/reprints?id=1-2A8YYHDZ&ct=220608&st=sb 12/16
5/25/23, 3:46 PM Gartner Reprint
Gartner’s Critical Capabilities research identifies and then analyzes the most relevant providers and their products in a market. Gartner uses, by default, an upper limit of 20
providers to support the identification of the most relevant providers in a market. On some specific occasions, the upper limit may be extended by Methodologies where
the intended research value to our clients might otherwise be diminished.

The inclusion criteria represent the specific attributes that analysts believe are necessary for inclusion in this research.

To qualify for inclusion, providers must demonstrate the capability to observe an application’s complete HTTP/S transaction behavior, either through proprietary agent
technology and/or distributed tracing. The vendor must demonstrate the capability to automatically collect data from at least three modern application frameworks:

Java Virtual Machines (JVMs)

.NET CLRs

PHP

Ruby

Node.js

AngularJS

Python

Go

The vendor must show at least three of the following core capabilities:

Automated discovery and mapping of application and its infrastructure components (including cloud services).

Monitoring of applications running on mobile (native and browser) and desktop browser.

Identification and analysis of application performance problems and their impact on business outcomes.

Native integration capabilities with automation and service management tools, as well as native integration with public cloud providers (for example, AWS Cloudwatch,
Azure Monitoring, Google Cloud Operations, etc.)

Analysis of business KPIs and user journeys (for example, login to check-out).

The ability to perform interactive interrogation of multiple telemetry types (that is, traces, metrics, logs) to detect “unknown unknowns”; that is, the ability to identify
underlying issues to unexpected events.

Application security functionality, delivered via a common agent or framework for APM.

Table 1: Weighting for Critical Capabilities in Use Cases

Digital Application
Critical IT Security SRE/Platform
Experience DevOps/AppDev Owner/Line of
Capabilities Operations Operations Operations
Monitoring Business

Business Analysis 0% 0% 20% 0% 90% 0%

Root Cause Analysis 25% 0% 25% 20% 0% 25%

ITService/Infrastructure 75% 0% 45% 25% 0% 25%

Monitoring

ADDP 0% 0% 0% 30% 0% 20%

RASP 0% 90% 0% 0% 0% 0%

Behavior Analysis 0% 10% 10% 25% 10% 30%

We use cookies to improve your experience

As of 1 June 2022
We use cookies to deliver the best possible experience on our website. To learn more, visit our Privacy
Policy. By continuing to use this site, or closing this box, you consent to our use of cookies. Cookie Notice.
Source: Gartner (June 2022)

https://www.gartner.com/doc/reprints?id=1-2A8YYHDZ&ct=220608&st=sb 13/16
5/25/23, 3:46 PM Gartner Reprint
This methodology requires analysts to identify the critical capabilities for a class of products/services. Each capability is then weighted in terms of its relative importance
for specific product/service use cases.

Critical Capabilities Rating

Each of the products/services has been evaluated on the critical capabilities on a scale of 1 to 5. A score of 1 equals Poor (most or all defined requirements are not
achieved), while 5 equals Outstanding (significantly exceeds requirements). However, readers are cautioned not to compare scores of the 2021 APM critical capabilities
ratings with those in the prior version (see Critical Capabilities for Application Performance Monitoring), because the evaluation process has been updated.

In the prior evaluation, Gartner developed persona-based scenario elements against which vendors were asked to demonstrate their product and/or service capabilities.
Thus, if the score in the current research is either higher or lower than in the prior version, you should not conclude that the vendor’s offering in some way performed better
or worse than last year.

Table 2: Product/Service Rating on Critical Capabilities

Amazon
Critical Alibaba Cisco
Web Broadcom Datadog Dynatrace Elastic Honeycom
Capabilities Cloud (AppDynamics)
Services

Business Analysis 3.2 1.9 2.7 3.7 3.8 3.8 3.8 2.9

Root Cause Analysis 3.1 2.8 3.5 3.3 4.0 4.2 3.7 3.2

ITService/Infrastructure 3.2 3.3 3.4 3.5 3.7 4.2 3.7 3.4

Monitoring

ADDP 3.4 2.6 3.3 3.2 4.0 4.2 3.8 3.8

RASP 3.5 3.3 2.9 4.1 2.4 3.3 3.0 1.0

Behavior Analysis 3.0 2.7 3.1 3.3 3.9 4.3 3.9 3.8

As of 1 June 2022

Source: Gartner (June 2022)

Table 3 shows the product/service scores for each use case. The scores, which are generated by multiplying the use-case weightings by the product/service ratings,
summarize how well the critical capabilities are met for each use case.

Table 3: Product Score in Use Cases

Amazon
Alibaba Cisco
Use Cases Web Broadcom Datadog Dynatrace Elastic Honeycomb
Cloud (AppDynamics)
Services

IT Operations 3.18 3.18 3.43 3.45 3.78 4.20 3.70 3.35

Security 3.45 3.24 2.92 4.02 2.55 3.40 3.09 N/A

Operations

Digital 3.16 2.84 3.26 3.47 3.82 4.13 3.74 3.29

Experience
Monitoring

DevOps/AppDev 3.19 2.84 3.32 3.32 3.90 4.23 3.78 3.58

We use cookies to improve your experience

We use cookies to deliver the best possible experience on our website. To learn more, visit our Privacy
Policy. By continuing to use this site, or closing this box, you consent to our use of cookies. Cookie Notice.

https://www.gartner.com/doc/reprints?id=1-2A8YYHDZ&ct=220608&st=sb 14/16
5/25/23, 3:46 PM Gartner Reprint

Amazon
Alibaba Cisco
Use Cases Web Broadcom Datadog Dynatrace Elastic Honeycomb
Cloud (AppDynamics)
Services

Application 3.18 1.98 2.74 3.66 3.81 3.85 3.81 2.99

Owner/Line of
Business

SRE/Platform 3.16 2.86 3.32 3.33 3.90 4.23 3.78 3.55

Operations

As of 1 June 2022

Source: Gartner (June 2022)

To determine an overall score for each product/service in the use cases, multiply the ratings in Table 2 by the weightings shown in Table 1.

Critical Capabilities Methodology

This methodology requires analysts to identify the critical capabilities for a class of products or services. Each capability is then weighted in terms of its relative
importance for specific product or service use cases. Next, products/services are rated in terms of how well they achieve each of the critical capabilities. A score that
summarizes how well they meet the critical capabilities for each use case is then calculated for each product/service.

"Critical capabilities" are attributes that differentiate products/services in a class in terms of their quality and performance. Gartner recommends that users consider the
set of critical capabilities as some of the most important criteria for acquisition decisions.

In defining the product/service category for evaluation, the analyst first identifies the leading uses for the products/services in this market. What needs are end-users
looking to fulfill, when considering products/services in this market? Use cases should match common client deployment scenarios. These distinct client scenarios define
the Use Cases.

The analyst then identifies the critical capabilities. These capabilities are generalized groups of features commonly required by this class of products/services. Each
capability is assigned a level of importance in fulfilling that particular need; some sets of features are more important than others, depending on the use case being
evaluated.

Each vendor’s product or service is evaluated in terms of how well it delivers each capability, on a five-point scale. These ratings are displayed side-by-side for all vendors,
allowing easy comparisons between the different sets of features.

Ratings and summary scores range from 1.0 to 5.0:

1 = Poor or Absent: most or all defined requirements for a capability are not achieved

2 = Fair: some requirements are not achieved

3 = Good: meets requirements

4 = Excellent: meets or exceeds some requirements

5 = Outstanding: significantly exceeds requirements

To determine an overall score for each product in the use cases, the product ratings are multiplied by the weightings to come up with the product score in use cases.

The critical capabilities Gartner has selected do not represent all capabilities for any product; therefore, may not represent those most important for a specific use
situation or business objective. Clients should use a critical capabilities analysis as one of several sources of input about a product before making a product/service
decision.

We use cookies to improve your experience

We use cookies to deliver the best possible experience on our website. To learn more, visit our Privacy
Policy. By continuing to use this site, or closing this box, you consent to our use of cookies. Cookie Notice.

https://www.gartner.com/doc/reprints?id=1-2A8YYHDZ&ct=220608&st=sb 15/16
5/25/23, 3:46 PM Gartner Reprint

© 2023 Gartner, Inc. and/or its affiliates. All rights reserved. Gartner is a registered trademark of Gartner, Inc. and its affiliates. This publication may not be reproduced or distributed in any form
without Gartner's prior written permission. It consists of the opinions of Gartner's research organization, which should not be construed as statements of fact. While the information contained in
this publication has been obtained from sources believed to be reliable, Gartner disclaims all warranties as to the accuracy, completeness or adequacy of such information. Although Gartner
research may address legal and financial issues, Gartner does not provide legal or investment advice and its research should not be construed or used as such. Your access and use of this
publication are governed by Gartner’s Usage Policy. Gartner prides itself on its reputation for independence and objectivity. Its research is produced independently by its research organization
without input or influence from any third party. For further information, see "Guiding Principles on Independence and Objectivity." Gartner research may not be used as input into or for the training
or development of generative artificial intelligence, machine learning, algorithms, software, or related technologies.

About Careers Newsroom Policies Site Index IT Glossary Gartner Blog Network Contact Send Feedback

© 2023 Gartner, Inc. and/or its Affiliates. All Rights Reserved.

We use cookies to improve your experience

We use cookies to deliver the best possible experience on our website. To learn more, visit our Privacy
Policy. By continuing to use this site, or closing this box, you consent to our use of cookies. Cookie Notice.

https://www.gartner.com/doc/reprints?id=1-2A8YYHDZ&ct=220608&st=sb 16/16