What to Measure: Structuring IT Metrics to Improve Performance
The question “what do I need to measure?” is a difficult one. There’s no standard framework or set of metrics that can be plucked off a shelf. Gathering and analyzing metrics takes time, so it’s important to get it right, otherwise you’re wasting resources. Measuring IT performance in the right way will help you IT align activity with what the business wants – and help you prove value in a way that the business understands. Measuring the wrong metrics can have the opposite effect – resulting in increased dissatisfaction with IT and a shrinking budget. Taking a strategic, structured approach to metrics will help you ensure you’re moving in the right direction.
Take a Top-Down Approach
Structure your metrics to support the sort of IT performance that the business wants to see. IT’s idea of good performance is largely irrelevant. Most often, IT’s idea of what great performance looks like will be very different from that of the business. In day-to-day operations, IT people need to continually ask themselves “Is what I’m doing of value to the business?” This mantra should also apply to the strategy for metrics: “Are these metrics helping us improve performance for the business?”
In order to understand whether metrics are indeed of value, IT needs to consult with business people. That means sitting down with business leaders to agree on what good looks like, and how you can measure against this. The output of this process should be a fairly small set of Key Performance Indicators (KPIs) that collectively form a well-rounded representation of IT performance from the business perspective. Too many numbers will put an excessive measurement overhead on IT and dilute the focus of improvement. Too few and the business (and IT) will not be clear on performance. Of course, this set of KPIs will differ from industry to industry and from business to business, but there are some common metrics for performance that IT can suggest.
Measuring IT customer satisfaction is a good example, as it essentially aggregates how happy business people are with what IT does. There are a number of methods for measurement, mostly adapted from general customer service metrics – including C-SAT and a variation on Net Promoter Score (NPS). What, how and when you measure will depend on what your organization needs, but this might involve:
- Making it easy for an end user to “rate IT” at any time they choose via a web portal.
- Asking for feedback after each interaction (or third/fifth/tenth interaction so as not to bother end users for feedback too frequently).
- Surveying end users across the whole organization on a monthly or quarterly basis to ensure a complete and accurate benchmark.
Generally, where tools are available to automate the collection of IT customer surveys, monthly is better – but if the process involves a lot of manual effort you will need to balance the desired frequency against what is realistically achievable.
Create a Logical Hierarchy of Metrics
Once you have a clear set of top-level KPIs you can start to work downwards to fill out a structured hierarchy of internal IT metrics that will help you focus improvement. Each one of these should tie directly to a top-level KPI. If you can’t identify how an internal metric links to overall IT performance, drop it.
Taking IT customer satisfaction as an example, you can define a sub-set of internal IT metrics that will help you shift the dial upwards. The performance of the service desk is the number one factor affecting IT customer satisfaction, so there are some common service desk metrics that can help guide improvements that will have a positive effect on the top-level KPI. For one, there is a proven correlation between the First Contact Resolution (FCR) rate and IT customer satisfaction ratings, so FCR would be a viable metric. Again, the number of metrics in the sub-set should balance the effort of measurement against the ability to focus activity on improvement. If you have too many internal metrics, you’ll spend too much time monitoring them, and it can be difficult to see where to start.
Form an IT Metrics Steering Group
Having a solid system of metrics is critical to IT performance, and this system needs to be managed over time to keep it on track. By forming a steering group – comprised of stakeholders from the business and IT operations – you can adapt your metrics model to ensure it remains fit for purpose in an ever-changing business environment.
The IT metrics steering group should review the hierarchy of metrics regularly – ideally every quarter. Business models change over time, so IT performance KPIs will need to change with them. This is just one way in which metrics go “stale”. If there is a fundamental change in the direction of the business (e.g. a merger or acquisition), this should be a trigger for a review. Also, as performance against internal IT metrics improves, the law of diminishing returns kicks in, so it may be time to restructure the metrics model to target more effective gains.
Steering group meetings present a good opportunity to discuss improvement projects tabled by IT – and the potential impact they will have on top-level KPIs. For IT this is a chance to demonstrate that they’re on the same page as the business and positive action is being taken to generate increasing value.
Another issue that the steering group must keep on top of is people trying to “game the system”. With the best will in the world, metrics often become counter-productive, particularly if there are bonuses and other rewards on the table. Just ask the banks. Metrics put people under pressure, so they naturally find ways to game the system. By altering the way metrics are gathered (or actually doctoring the results) it is possible to output more favorable numbers – and the story that the metrics tell can get further and further from the truth. The steering group should set policies for how metrics should be gathered and communicate clearly the penalties for rigging the numbers. Spot checks and audits may be ordered to ensure things are kept on the straight and narrow. Without trustable metrics, the whole system can quickly collapse, so ensuring honesty is a critical success factor.
Good metrics help IT align activity with what the business needs. And reviewing them regularly will help IT stay synchronized when business objectives change – as they inevitably do. Done right, metrics can be a strategic business tool that brings IT and the business together to optimize the value that IT delivers. Getting the right people together (from IT and the business leadership) to devise and manage a strategic approach to metrics will pay dividends by ensuring sure IT is always on the path that is most productive for the business.
You may also want to check out: For Heaven’s Sake Please Take The Meh Out Of Your Service Desk Metrics
Posted by Joe the IT Guy