View All Videos

How to measure the impact of IT operations on your business (Part 2)


Damon Edwards / 

Part 1: Putting a metrics/KPI program into place in 6 steps 

Part 2: Identifying candidate KPIs to evaluate

In my first post in this series, I went through the six steps for putting into place a metrics/KPI program that measures the performance of your IT operations within the context of your business goals.

When consulting, this is usually the point where I stress that we have to work the process in order to come up with KPIs that mean something to your specific business. I explain that there is no such thing as one size fits all in this matter. Despite that, the very next question I’m usually asked: “Can you tell us now what KPIs a company like ours should be measuring?”

Just providing a list of example would probably send them off on the wrong course by chasing KPIs that were important to someone else’s business. Since figuring out what to measure can be as valuable as the actual measurement, I instead walk them through the following concepts to get them started on step #2 and step #3 of the process.


First, stop and consider what “measurement” really means

Measurement: a set of observations that reduce uncertainty where the result is expressed as a quantity


I lifted the above definition from measurement guru, Douglas W. Hubbard. However, if you noodle around in the academic writings on this topic, you’ll see that it’s a fairly accepted definition.

When looking for a way to measure something, keep this definition in mind. Whether its problem solving, allocating budget, or prioritizing your resources, reducing uncertainty gives you a decisive and valuable advantage. You don’t need to have absolute precision. A coarse swing at something is often going to be enough to get started reducing uncertainty and providing business value.

Don’t forget to consider that not every measurement has to be expressed as a simple number (e.g. “137 occurrences” or “83.2% of the time”). You can measure things on an ordinal scale (e.g. “this is less than that” or “this gets 3 out of 4 stars”). You can use nominal measurements where you are are only considering membership in a set (e.g. “this action is in category x, that action is in category y”). Yes/No questions are a valid kind of measurement. You should even consider using subjective methods of measurement (e.g. “do you feel this week was better than last week?”).

Also, don’t expect that every measurement will be made at the same time interval. Sometimes it makes sense to measure certain things on a daily basis. Sometimes it makes sense to measure other things on a quarterly basis.

No matter what type of measurement you end up employing, make sure that it is clear to everyone — even the casual observer — how and why you are measuring something . This is critical for gaining buy-in and avoiding gaming (which both seem like excellent topics for future posts in this series!)


Then use “The Four Buckets” as a guide to start looking for candidate KPIs

At the end of the KPI development process, you are going to be tracking a small set of KPIs that best measure the performance of you IT operations in it’s role supporting your business’s goals. But to get there, you need to start with a larger pool of candidate KPIs. In my experience, most useful measurements tend to fall into one or more of the following categories.

I call these “The Four Buckets”.

Again, keep in mind that at this stage you are looking to surface possible KPIs that will be feed into the rest of the process. The end result will only be a small subset of what you started with (5 – 10 at the most!)


1. Resource Utilization – How resources are allocated and how efficiently they are used. Usually we’re talking about people, but other kinds of resources can fall into this bucket as well.

  • How much time do developers and administrators spend on build and deployment activity?
  • How much productivity is lost to problems and bottlenecks? What is the ripple effect of that?
  • What’s the ratio of ad-hoc change or service recovery activity to planned change?
  • What’s the cost of moving a unit of change through your lifecycle?
  • What’s the mean time to diagnose a service outage? Mean time to repair?
  • What was the true cost of each build or deployment problem (resource and schedule impact)?
  • What percentage of Development driven changes require Operations to edit/change procedures or edit/change automation?
  • How much management time is spent dealing with build and deployment problems or change management overhead?
  • Can Development and QA successfully deploy their own environments? How long does it take per deployment?
  • How much of your team’s time is spent recreating and maintaining software infrastructure that already exists elsewhere?


2. Failure Rates – Looking at how often processes, services, or hardware fail is a pretty obvious area of measurement.

  • What was the ratio of successful builds to failed or problematic builds?
  • What is the ratio of build problems due to poor code vs poor build configuration?
  • What was the ratio of successful deployments to failed or problematic deployments?
  • What is the ratio of deployment problems due to poor code vs poor deployment configuration or execution?
  • What is the mean time between failures?


3. Operations Throughput – The volume and rate at which change moves through your development to operations pipeline.

  • How long does it take to get a release from development, through testing, and into production?
  • How much of that is actual testing time, deployment time, handoff time, or waiting?
  • How many releases can you successfully deploy per period?
  • How many successful individual change requests can your operations team handle per period?
  • Are any build and deployment activities the rate limiting step of your application lifecycle? How does that limit impact your business?
  • How many simultaneous changes can your team safely handle?
  • What is business’ perceived “wait time” from code completion to production deployment of a feature?


4. Agility – This looks at how quickly and efficiently your IT operations can react to changes in the needs of your business. This can include change driven by internal or external business pressures. There is often considerable overlap with bucket 3, however this bucket is focused more on changing/scaling processes than it is on the throughput of those processes once in place. (Of course, you can always argue that all four buckets play some role in enabling a business to be more “agile”.)

  • How quickly can you scale up or scale down capacity to meet changing business demands?
  • What’s the change management overhead associated increasing/decreasing capacity? What’s the risk?
  • How quickly and what would it cost to adapt your build and deployment systems to automate any new applications or acquired business lines?
  • What would it cost you to handle a x% growth in the number of applications or business lines (direct resource assignment plus any attention drain from other staff)?
  • Could your IT operations handle a x% growth in number of applications or business lines? (i.e. could it even be done?)



55 Responses

  1. Greg says:

    Great post. I appreciated the insight into ops teams needing to understand their roles/abilities within an agile release environment, as well as the need to also focus on scaling up and scaling down of capacity. These are new challenges facing IT operations, particularly in a world of complex and highly distributed applications.

  2. Suyas says:

    Wonderful information. I will go to share this to my friends…

  3. Using our Virtual Offices in London is very easy and signing up couldn’t be easier. Our competitive pricing is maintained to suit our clients needs.

  4. The Chartered Institute of Public Finance and Accountancy (CIPFA) is the leading accountancy body for the public services providing education and training in accountancy and financial management.

  5. Charter Selection is a leading recruitment agency made up of highly experienced industry professionals with a vast amount of experience in their specialist fields.

  6. Burton Sweet Corporate Recovery and Insolvency Practitioners

  7. ALEX says:

    The Authority has finalized its formal response to the Framework Implementation and Impact Study and its recommendations. assignment help uk .This response sets out the main areas in which work is already underway, such as on learning outcomes and the recognition of qualifications, and those areas in which specific or additional efforts are now required to meet priority needs, such as in relation to the labor market, standards development, the recognition of prior learning and a re-visiting of progression and the effective comparability of awards at Levels 4-6.

  8. Botox Agoura Hills is definitely the best gift you can provide to your skin and body. The choice of thousands of people can be the reason for the return of your lost glory of your beauty. Go ahead to help yourself.

  9. Binary Options are unique in relation to stocks, securities and shared stores; even along these lines, they are very easy to get it.

  10. I Like that kind a nice Blog.

  11. Thank you! It was a wonderful chance to visit this kind of site. I hope you will publish more on this topic. Thanks a lot for sharing with us!

  12. Anna Max says:

    An extraordinary advantage to open source tasks is the capacity to send force solicitations, enhancing the things one doesn’t care for. write my essay assignment for me | I generally take this open door as a risk to enhance the task and myself.

  13. Dennis says:

    Any Medicare Coverage, generally has its own set of conflicts and this is no other.

  14. Quinton Spence says:

    You may like checking the URL that will relieve you from all anxieties pertaining to your perfect Blue Cross Blue Shield plan.

  15. Chris Barrera says:

    Great post. Really I am surprised about this post.Thank You very much. Εξωσωματική Γονιμοποίηση

  16. Albert Wilder says:

    sack trucks is extremely economical on pocket and hence can be easily bought by one and all for every purpose.

  17. Are you looking for hottest drone with camera? You are at right place where you can find different types of drone with camera. You can select drones that are suitable to your requirement.

  18. Nba Curry says:

    NBA superstar Stephen Curry is simply one of the most talented and determined players in NBA history. Curry has made his way to being one of the top NBA players and is renowned for his awesome ball-handling skills.

  19. Young and talented is just but an understatement. Ronaldo R Madrid currently boasts being nominated the best player in the European UEFA league 2015.

  20. The versatile maternity jeggings leggings is a must for your wardrobe
    These are the maternity leggings from 24/7 comfort apparel that features an elastic waistband and its soft constructed for an easy and comfortable fit.

  21. Chiefâs and senior administrators need to see how their kids and more youthful grown-ups are caught up with skimming on the Internet looking for one stuff or the other so Lagos Nigeria Internet Marketing Lagos Nigeria is developing in minutes.

  22. Nile cruise says:

    The Go Nile cruise is no small name when it comes to throwing something good at its clients.

  23. Drop shipping has largely aided the growth of online stores. Thanks to it, clients can buy products online and have them delivered at their doorstep.

  24. If you are staying in Brisbane, then you can go for Rent a skip bin brisbane south as they are reliable contractors and make sure that your house is provided with the benefits that it needs.

  25. happy wheels says:

    I am very sympathetic to your viewpoint. It is very deep and meaningful. I think you should write many more articles to the reader to understand. I would recommend it to everyone.

  26. Great. This article is excellent. I have read many articles with this topic, but I have not liked. I think I have the same opinion with you.

  27. Equally important is choosing an appropriate package that suits your needs. However, choosing Wedding photography packages sydney is not that easy.

  28. RemoveWAT 2.2.6 or Best Windows 7 Loader may fix your all windows 7 ultimate, windows 7 sp1 and windows 7 sp2, business, windows 7 professional, windows 7 home, and windows 7 simple edition. It is actually 100% working along with tested by everyone.

  29. Muller Travolta says:

    Such a great post. Take a look at our beneficial reviews on assignment writing from different sites to get reliable analysis of academic websites.
    Assignment Writing Reviews for You

  30. Despite from the durability as well as longevity, marble tiles are often stained. Since the case, anything made associated with marbles demands regular treatment and cleansing. However cleansing of marble isn’t same because that associated with cleaning associated with regular flooring.

  31. Despite with the durability and also longevity, marble tiles can be stained. Because the case, almost everything made regarding marbles needs regular attention and washing. However washing of marble just isn’t same since that regarding cleaning regarding regular surfaces.

  32. Despite in the durability along with longevity, marble tiles are easily stained. Because case, something made involving marbles calls for regular proper care and cleanup. However cleanup of marble is just not same while that involving cleaning involving regular flooring surfaces.

  33. Despite on the durability in addition to longevity, marble tiles are often stained. For the reason that case, any item made connected with marbles involves regular health care and clean-up. However clean-up of marble seriously isn’t same seeing that that connected with cleaning connected with regular floor surfaces.

  34. Despite of your durability plus longevity, marble tiles can be stained. As being the case, anything made with marbles necessitates regular caution and cleaning up. However cleaning up of marble will not be same when that with cleaning with regular floorings.

  35. aladdinsup says:

    Mediterranean Supermarket Miami
    From carefully selected skinless and boneless chicken portions, to delicious, ready-to-cook lamb burgers, there’s always a great time to try Supermarket

  36. a10 says:

    You’ve written nice post, I am gonna bookmark this page, thanks for info. I actually appreciate your own position and I will be sure to come back here.
    color switch

  37. roblox says:

    I really like the dear information you offer in your articles. I’m able to bookmark your site and show the kids check out up here generally. Im fairly positive theyre likely to be informed a great deal of new stuff here than anyone
    mortal kombat x

  38. Falafel5 says:

    Excellent middle eastern food.
    Serves great Halal Restaurant Miami food. Location is great. I wish there were more diners like this one.

  39. ppm-factory says:

    I adore an incredibly good article just like it. You actually think nearly the same as I actually do and I am pleased I found your article.

  40. Bobby says:

    The huge cost of IT is going to put a big hole in my business account. I think we should talk to Brian Gaister so that we get more funds for this.

  41. kesehatan semakin kaki pemilik lapis dapat sangat akan dapat keempat untuk di itu kontraktor untuk
    konstruksi baja wf

  42. for more new year 2017 messages, hd images and wallpapers.,

  43. results says:

    get your online result here

  44. Thank you for the sensible critique. Me and my neighbor were just preparing to do a little research on this.

  45. thanks it mean a better bad advnace tool for me

  46. WatsonZion says:

    To do your tax return more than three times what you realize requested amount of money away in RRSP Write my assignment
    important now because the government is sponsoring that your program has done a good job, and you start thinking, “How full-time artists do so

  47. pioneerseo says:

    Yes, extraordinary US Military drive. Likewise, in his post you have allowed to tune in about US Military. I truly value your work. A debt of gratitude is in order for sharing tourism and implantation information

  48. IT is the vital role in our business, i am teacher and helping the students whose are fed up with their law essays projects.

  49. WatsonZion01 says:

    I set out, Reliable Essay Writing Service which is not available as nodes in the whole thing a bit.

  50. belli jons says:

    microsoft support number always provide support of Microsoft customer and provide the solution at a minimal price.

Post a new comment