Metrics 101: Implementing a Metrics

Framework to Create Value through

Continual Service Improvement
By David Smith

It’s often been said that “you can’t manage what you don’t measure,” which is true. Without
purpose and a course to follow, the destination is uncertain and almost always unpredictable.
Many management books have been written on this subject, ranging from personal development
to organizational leadership. They all agree in principle that a purpose, goal, or destination must
be determined in order to chart a course and path to achieve them. Once the path or road map
has been defined, the journey must be carefully planned to guide the traveler safely to the desired
destination in the prescribed time within planned costs.

Measurements are like navigational aids. They help identify the destination, the road map to follow,
hazards to avoid, milestones to reach, fuel consumption, constraints or limitations, expected
time of arrival, and so on. Without navigational aids, one could get lost, end up anywhere, get
stranded, fall off a cliff, run out of fuel, get in an accident, or fall asleep at the wheel.

The goal of this article is to provide the reader with a flexible and scalable measurement framework
that is easy to learn, implement, manage, and improve. This framework, based on a continual
improvement lifecycle, is intended to provide process metrics and techniques to help align IT
with the business objectives in order to create value by making processes and services more
“efficient and effective.” It helps the reader determine ways to:

• Align it with business objectives and verify the results;

• Maintain compliance requirements for business operations; and
• Drive operational efficiencies, effectiveness and quality.
The measurement framework can be implemented as a comprehensive measurement program for
all processes and services, or selectively for individual process or services. It is aligned with ITIL,
also a set of best practices, and it is compatible with COBIT and supports ISO/IEC 20000.

1
More details can be found in the book Implementing Metrics for IT Service Management, which
provides methods, concepts, examples, techniques, checklists, and software templates for
accelerating adoption using a “how-to” based approach.

What Metrics Are All About

Metrics define what is to be measured. For IT, this includes technology, processes, and services.
These measurements provide a feedback mechanism that enables management to steer, control,
and guide IT toward strategic objectives. Metrics help to:

• Align business/IT objectives:

»» Accounting of IT processes and deliverables,
»» Inform stakeholders,
»» Understand issues, and
»» Influence behavior;
• Achieve compliance:
»» IT operations strategy,
»» ISO/IEC 20000, COBIT, service levels,
»» Critical success factors, and
»» Minimize interruptions;
• Establish operational excellence:
»» Measure, control, and manage cost effectiveness,
»» Improve effectiveness and quality,
»» Service level improvements, and
»» Maximize value creation.

Implementing Metrics

ITSM metrics must measure process and service effectiveness, as well as the functions and
technologies that provide them. Metrics in IT have traditionally been measured in functionally-
oriented silos like the help desk, server technical services, or the operations department. IT
departments are shifting to process- and service-centric organizational models requiring metrics
that report beyond the functional boundaries to determine success. For example, both the
application development and IT operations departments are functionally very mature and, when
independently measured, appear successful. However, they don’t work well with each other and
together frequently fail to deliver deployments.

2
Metrics have been very mature for measuring technology availability on a discrete component
basis, but in many cases without consideration for the end-to-end user experience. For example,
the application server was available 99.99 percent of the time, but the network is not measured
and is frequently unavailable or unresponsive. Therefore, the measure of system availability
(server plus network) does not match the user experience.

A new-and-improved approach for implementing metrics using a continual improvement

framework is needed to align IT to the business. It must meet new and changing compliance
requirements and provide a means of gaining operational excellence. As outlined in Implementing
Metrics for IT Service Management, this type of “measurement framework” reference model, which
can be quickly implemented, adapted, and evolved to meet the organization’s needs, has several
key features:

• Continual improvement (that is, W. Edward Deming’s Plan-Do-Check-Act cycle);

• Top-down design approach for aligning goals and objectives;
• Process- and service-based IT service management approach;
• Scalable and flexible fit-for-purpose model with hundreds of sample metrics and
scorecards;
• Bottom-up reporting of facts, metrics, indicators, scorecards and dashboards;
• Aggregation of metrics to formulate key performance indicators;
• Accountability and role-based matrix models;
• Techniques for comparative, causal, and predictive analysis
• Method for filtering improvement initiatives and tracking performance status;
• Ability to report performance improvements and derived-value-based benefits;
• Multiple implementation methods and scenarios;
• How-to checklists for planning and implementing metrics; and
• Scorecard accelerator templates to demonstrate principles and techniques and help
kickstart a measurement program implementation.

3
Basic Concepts

There are four critical success factors for an effective measurement framework:

• It enables validation of the strategy and vision by:

»» Aligning with the IT goals and objectives,
»» Validation that alignment is working, and
»» Confirm goals and objectives are met.
• It provides direction with targets and metrics, by:
»» Setting targets through metrics,
»» Controlling and managing the processes, and
»» Verifying that targets are being met.
• It provides a means to gauge value realized, by:
»» Justifying performance improvements with a solid fact base,
»» Quantifying benefits realized, and
»» Communicating the value realized with factual evidence.
• It enables intervention and corrective actions by:
»» Identifying deviations when they occur,
»» Understanding the root causes, and
»» Intervening with corrective actions to minimize consequences.

The Measurement Process

The measurement framework is broken down into a series of processes, which, in turn, are
composed of four main subprocesses that repeat to form a continual improvement feedback loop
(based on the Plan-Do-Check-Act cycle). The subprocesses of the measurement process are:

1. Tuning (Plan): The tuning subprocess is responsible for identifying improvement

opportunities and making recommendations for the subject process or service being
measured. Note that the tuning subprocess can also act as the entry point for planning
the measurement program and framework.
2. Implementation (Do): The implementation subprocess is responsible for implementing
the recommended changes through normal change management processes. Note that
the implementation subprocess can also act as the entry point for implementing the
measurement program and framework.

4
 3. Monitoring (Check): The monitoring subprocess is responsible for gathering data,
making calculations, and validating the required measurements.
4. Analysis (Act): The analysis subprocess is responsible for comparative, causal, and
predictive analysis to determine what corrective actions may be required.

There are two additional supporting subprocesses that provide administration and reporting:

5. Administration: This subprocess is responsible for the administration of the activities

associated with the maintenance of the metrics and measurement database (MDB).
6. Reporting: This subprocess is responsible for reporting the findings and recommendations
to management and various stakeholder groups, keeping them informed and aware.

The outputs of the measurement process are used to report the status, findings, and
recommendations of various service management processes and services to key stakeholder
groups within the organization. Some of these are as follows:

• Process- and service-based performance reports;

• Exception handling reports;
• Notices and alerts;
• Root cause analysis and observations;
• Predictive analysis and observations;
• Change requests;
• Status of new and existing service improvement initiatives; and
• Benefits or value derived from processes, services, service assessments, audits, and
reviews.

Benefits

Measurements help improve performance, align goals, and realize value. The positive benefits can
be weighed against the negative consequences of not having a measurements program.

The benefits of a measurement program include the following:

• Provides the instrumentation necessary to control an organization;

• Direct focus on specific performance and control objectives;
• Easier to spot danger in time to correct it;
• Improves morale in an organization;
• Stimulates healthy competition between process owners;

5
 • Helps align IT with the business’s goals and verify results;
• Drives efficiency, effectiveness, and quality;
• Inspires continual improvements; and
• Helps reduce total cost of ownership (TCO).

Some of the negative consequences of not having a measurements program include:

• Reduced visibility resulting in loss of control;

• Focus on “noise” instead of what’s important;
• Reactive fire-fighting mode;
• Low morale in organization;
• Unhealthy political competition;
• Benefits not apparent or realized;
• Cost effectiveness not understood;
• Customer complaints drive improvements;
• TCO not optimized; and
• Increasing risk.

Summary

Implementing a measurement framework should help align IT with the business objectives and
create value through continual improvements. The measurement framework acts as the road
map; the business’s goals and objectives are the destination, critical success factors provide the
directions, and metrics provide the sign posts to keep you on course. Whether implemented as a
comprehensive measurement program for all processes and services, or selectively for individual
process or services, each organization can use this approach and the techniques discussed to
create its own tailor-made measurement framework for improving its performance.

David Smith is the president of Micromation, Inc., which specializes in benchmarking, metrics, TCO,
ISO 20000, and ITSM implementations. He is the author of Implementing Metrics for IT Service
Management and contributing coauthor of IT Service Management: Global Best Practices (both Van
Haren Publishing, 2008).