View All Videos

How to measure the impact of IT operations on your business (Part 1)

59

Damon Edwards / 

Part 1: Putting a metrics/KPI program into place in 6 steps

Walk into any web-based business and more likely than not you’ll find all sorts of metrics that have been collected with varying levels of accuracy, consistency, and freshness. However, despite what appears to be a wealth of data, no one will seem to be all that happy with it.

The frontline guys are grumbling about having to deal with the collection and reporting overhead while questioning under their breathe “what’s in it for me?”. The technology managers are still making decisions based on anecdotal evidence, gut feelings, or knee-jerk reactions to the latest incidents. And to top it all off, the business managers are clamoring for new dashboards because the last ones didn’t tell them much that was actually useful for managing the business.

In short, there is endless data floating around, but no one appears to have the knowledge they want.

 

 

Invariably another “metrics project” is going to come down the project pipeline. But why would this effort turn out any different than the previous efforts?

Here are six straightforward steps for putting into place a metrics program that actually delivers knowledge about your operations and your business:

Step #0: Stop looking for metrics and start thinking about KPIs
Most metrics projects that are initiated within a technology organization will take a bottom up view of the world. The thought process is often to grab as much data as possible and then try later to make some sense of it with a bit of analytical magic. The dream is that, with enough technical mojo, you’ll arrive at a magical “ah ha!” moment where a piece of important knowledge jumps out.

Unfortunately, for as common as this dream is, it rarely comes true. After considerable effort, these bottom up metrics projects usually result in an outcome somewhere between a mix of interesting (but not meaningful) trivia or a loss of interest in the project alltogether.

To ensure success, you’ve got to turn your technology-centric view of this problem on its head. Since the role of IT is to support the goals of the business, it logically follows that what you really want to measure is the performance of your IT operations in supporting those business goals. This is done by developing and tracking a set of Key Performance Indicators (KPIs) that align the performance of your IT operations with the performance of your business

What’s the difference between metrics and KPIs? The following list is the best explanation I’ve seen on how to tell if a metric (or set of metrics) qualifies as a KPI :

  1. A KPI echoes organizational goals
  2. A KPI is decided by management
  3. A KPI provides context
  4. A KPI creates meaning on all organizational levels
  5. A KPI is based on legitimate data
  6. A KPI is easy to understand
  7. A KPI leads to action!

KPIs, by their very nature, are about influencing actions. If a metric isn’t capable of influencing the behavior of your team in a way that would be clearly understood from all directions, then it’s not a KPI.

Step #1: Set up a KPI Advisory Board
KPIs, like all metrics, have little value in isolation. What’s the best way to ensure that your KPIs are going to be successful in measuring and influencing the performance of your organization? Get all of the various stakeholders involved early and often.

As your very first step, create a KPI Advisory Board that is comprised of key stakeholders from each part of your organization (Dev, Ops, QA, Business/Product Management, Finance, etc). The KPI Advisory Board is responsible for validating/updating your KPI choices, overseeing data collection, analyzing results, and relaying results to the rest of the organization. The KPI Advisory Board also helps to keep everyone honest and avoid intentional or unintentional gaming of the KPI process. Make sure the KPI Advisory Board meets on a regularly scheduled basis. Don’t overload the meetings with other issues like architecture or budgeting. Stay focused on organizational and process performance. If the KPI Advisory Board is a burden on anyone, it’s probably not being run correctly.

Due to the human dynamics involved with assembling a KPI Advisory Board, may technologists feel compelled to skip this step. DON’T. The KPI Advisory Board is essential to the success of your KPI program. Reaching consensus on what to measure and how to measure it is as important to the health of an organization as the actual measurement.

 

Step #2: Prioritize what is important for your business
First, get a good understanding of how business performance is measured within your company. Every business should already know what indicates good business performance in their specific case. Every business should also know what it’s strategic goals are. Ask around if you don’t know. Better yet, if you’ve formed your KPI Board correctly, those answers will come to you via the members from other parts of your organization. Be sure to take note of the priority/weighting of each goal (this will sometimes vary depending on who you asked).

Next, create a list of the ways that IT operations can impact/support each of those business goals. Your specific list will obviously depend on your business’s specific goals. However, you’ll likely find that there are four general buckets of IT operations objectives that can be mapped to your business’s goals (I’ll dig into each of these in a future post):

  • Improve resource utilization/efficiency
  • Decrease failure rates
  • Improve operations throughput
  • Enable business agility

What you should now have is a list of specific IT operations objectives mapped to prioritized business goals.

Side note: Beware of falling into the trap of thinking that what matters to the individual members of your technology team is the same as what matters to the business. Sure, less individual headaches is a good thing because headaches are generally a sign of larger systemic issues. But your business representatives on the KPI board should push back and remind you that solving your individual headaches many not be an important factor in the success of the business (and therefore not a KPI priority).

 

Step #3: Weight possible KPIs against your prioritized IT operations objectives
Work your way through the list of IT operations objectives and list out all of the possible candidate KPIs for each objective. Be sure to cast a broad net. Look through old metrics projects. Ask around your organization (“How would you measure…?”). Look at what others have published (“Google is your friend”).

I’ve found that it handy to keep Douglas Hubbard’s “Four Useful Measurement Assumptions” in mind when searching for candidate KPIs:

  1. Your problem is not as unique as you think
  2. You have more data than you think
  3. You need less data than you think
  4. There is a useful measurement that is much simpler than you think

Most importantly, you need to continuously ask the question “Is this really important?”. Just because something is interesting doesn’t mean it’s valuable.

Go through the list with your KPI Advisory Board. Weight the effectiveness of each candidate KPI for indicating the success/failure of each prioritized IT operations objective. Also, be sure to weight the “difficulty of measurement” for each candidate KPI. While you will ultimately be concerned with a KPIs ability to indicate progress towards meeting goals, understanding the “difficulty of measurement” can have practical implications when deciding where to focus first.

Step #4: Gain consensus on a manageable set of KPIs
Work first within the KPI Advisory Board and then across your broader organization to gain consensus on a manageable set of initial KPIs. I’d recommend that your start with no more than 5 – 10 KPIs. Keep it simple. Get the win.

To select your initial set of KPIs use the weighting and prioritization determined in the previous steps. More often than not, you’ll be surprised at what candidate KPIs come out on top of the list. Of course a bit of gut feeling and pragmatism may come into play, but try to trust the prioritization process. If company politics or strong interpersonal dynamics get in the way, try your best to still reach consensus (but stay focused on getting something done).  Your initial goal should be to gain experience with the process and to achieve early success that the organization can buy into. You can always expand and/or refocus later.

Step #5: Baseline and track
Create a baseline for each of the initial KPIs you’ve selected. This is also a good time to validate any of the assumptions you’ve made in the previous steps.

If everything looks reasonable, begin regular tracking of the initial set of KPIs. It’s best to start with manual (or semi-manual) data collection and KPI reporting. If you jump straight for a tool you are going to get caught up in what can/can’t be automated and what the right way is to go about that automation. This will just distract from what you are really trying to figure out… are we tracking the right thing and does the output make sense?

Transparency is an important issue in this process. Make sure that anyone with a reasonable understanding of your operations is able to understand how and why these KPIs indicate success/failure (e.g. “Can the CEO/CFO make sense of this?”)

Step #6: Re-evaluate and expand
Your KPI Advisory Board should meet on a regular basis to validate/analyze results and propose new KPIs (if any). Initially these KPI Advisory Board meetings should be held weekly.  However, over time, monthly KPI Advisory Board meetings (with weekly or daily distribution of KPI reports) will usually be sufficient.

Also, this is the point where you should start investing in automating the data collection and KPI reporting. Once you get a few cycles under your belt, you’ll have an understanding of what needs to be measured and how it should to be measured. And perhaps more importantly, you should have earned the buy-in and budget approval needed to get automated data collection and KPI reporting done correctly. Automation will ultimately allow you to track more KPIs (not always a good thing) with a higher degree of accuracy (a good thing) and provide a quicker feedback loop for your organization (always a good thing).

How difficult is it to implement a KPI program?
This is a common question. My consulting company helps companies accelerate the building of their KPI capabilities. We’ve got a mature methodology (based on the steps above). We use decision modeling tools built by partners. We use proven practices for building consensus and extracting analysis. We get our clients to a mature process as quickly as possible. Next to an example like that this may seem overwhelming to tackle on your own, but it’s really not.

KPI programs might be difficult to perfect, but they are easy to start. Start small, focus on building consensus, and grow iteratively. Since all businesses are dynamic, refining your KPI capabilities is more important than any one measurement.

 

Go to Part 2 in this series –>

 


59 Responses

  1. If you spring plain for a implement you are going to gain caught up in what can/can’t be automated moreover what the fit boulevard is to go about that automation.

  2. tinder pc says:

    Looking for Girls and boys to date tinder app is all you need you can grab tinder on computer for windows system

  3. subaedah says:

    situs yang berisi mengenai informasi seputar bagaimana cara memakai jilbab

  4. Lonesome says:

    Anthony,Yes it means its having pmlboers connecting to the MSN Adcenter servers. Are you using a valid MSN Adcenter login? That should work!

  5. Your first ambition should be to accomplish skill beside the prepare also to attain betimes fame that the alliance can bribe against. You can always stretch also/or refocus after.

  6. That probably explains why so many metrics fall short on delivering a comprehensive and reliable reporting process, the fact that metrics projects are initiated within technology organization usually taking a bottom up view of the world.

  7. We now have analysis your write-up, We are a great deal intrigued if you strategy for purpose incredibly superb plus educative.

  8. manisu says:

    frontline guys are grumbling about having to deal with the collection and reporting overhead while questioning under their breathe “what’s in it for me?”. The technology managers are still making decisions based on anecdotal evidence, gut feelings, or knee-jerk reactions to the latest incidents. And to top it all off, the business managers are clamoring for new dashboards because the last ones

  9. you can get mobdro for pc as well

  10. happy wheels says:

    Great. This article is excellent. I have read many articles with this topic, but I have not liked. I think I have the same opinion with you.

  11. Really great post, Thank you for sharing this knowledge.

  12. animal jam says:

    Excellently written article, if only all bloggers offered the same level of content as you, the internet would be a much better place. Please keep it up!..

  13. This excellent topic has long been among the finest subjects to find out you just read concerning. I have found your own write-up to be really rousing besides abundant in great details. I will look at your own further information immediately.

  14. The Nintendo 64 likewise has numerous asset hustling recreations that numerous individuals will appreciate.

    runx

  15. Investigate the sorts of amusements you need, the amount you need to spend and afterward you can pick a console that will be fun and the general population you play with.

    gamehere

  16. Greg Ahen says:

    Business operations are a key component of any industry and it has been explained better here by your post. I am running a small business.
    thesis

  17. Individuals prior merely kinds close friend grew to be made available fighting pertaining to some kind of agonize like this!

  18. It’s contains very useful data which i need and i want to see more quality posts in this blog so please update your blog. Thanks for sharing.

  19. Hi and thanks for the blogs. I liked the reading through the blog. the pictures in the blogs were awesome.Thanks for sharing such an awesome idea.

  20. Satisfying aspects.. i must say i specific and also your blog, maintain making!

  21. color switch says:

    I really like the dear information you offer in your articles.

  22. I’m able to bookmark your site and show the kids check out up here generally.

  23. Thank you for this! I’m working on a massive site redesign and these tools will really help. Tinder For PC

  24. dji italia says:

    This post is very simple to read and appreciate without leaving any details out. Great work! You completed certain reliable points there. I did a search on the subject and found nearly all persons will agree with your blog

  25. We are aware about the ways design has transformed. You can expect a touch of class in design projects assigned to us. The idea is to help visitors to traverse through design. We work on concepts that showcase and capture the imagination of the audience.
    Web Development in Dubai

  26. gmail says:

    free gmail download here.

  27. I have to declare my partner and i revered this kind of Adventure inside the Gem stone.

  28. MAX says:

    Enroll your German shepherd dog puppy in an obedience class as soon as possible, and train your dog thoroughly in basic obedience. …
    Teach your German shepherd dog to cease barking on command. …
    Encourage your dog’s bark announcing strangers who come to your house.

    Alternatives to Showbox
    showbox alternatives

  29. TSG 250 says:

    Brilliant write-up. When i get pleasure from. Appreciate it intended for write about of which at ease with you.

  30. I could discover great information from your site articles.

  31. I have to declare my partner and i revered this kind of Adventure inside the Gem stone.

  32. totally free Video calling and messanger app that allows you to connect with friends and family easily.
    Communications app for free text, video calls, moments, timeline and many more.
    CONNECT TO EASY

  33. I am very happy know this thanks for sharing this

  34. Walking Dead is special television series

  35. That is really amazing. Thank author for it is rare articles. Thanks for putting in the effort to put it all together.Thank you for new issue, I really like this website.

  36. First of all certainly, exhibiting a remarkable site. I’ll be considering searching for more connected with such a materials in addition to might probably wish to obtain more info.

  37. Exceptionally user friendly website. Tremendous details readily available on couple of clicks.

  38. how to recover icloud password?

    create icloud account

  39. Priyanka says:

    Thank you for this informative user-friendly Blog. I’m very satisfied to you for creating this. Many Many Thanks

  40. SHARE it says:

    If you spring plain for a implement you are going to gain caught up in what can/can’t be automated moreover what the fit boulevard is to go about that automation.

    IMO Video Call
    Mortal Kombat Annihilation Full Movie

  41. Xiaomi Mi6 says:

    According to read this post… I knew more info..

  42. Vape India says:

    Evolve Vapors as the name suggests has really made me Evolve into a better person. From helping people to Quit cigarettes and attracting people into the real world of Vaping, Evolve has done a great job! Though Vaping is growing faster and there are other ports where we can get our Vaporizers but what Evolve gives is different. Way different. It’s quality. and value for money.
    Evolve is Education and is educating people in the most righteous manner possible. I mean who will do that? It’s just that Evolve really cares about what its doing, about creating value. I’d say, blindly trust in Evolve and they’d show you the right path.

    Vape India,
    Vape Pen,Vaping India

  43. Alexandra Brisbane says:

    When you face connection problems while setting up your router, you want to access the configuration settings of your network. You want to make sure that hardware is not involved with any faults. Device manager or event log will help you to get rid of these problems in an effective manner. Even, you can make use of some useful tools to make sure about the computer’s network configuration ip 192.168.1.1.

  44. eva air says:

    If you spring plain for a implement you are going to gain caught up in what can/can’t be automated moreover what the fit boulevard is to go about that automation.

  45. Very well article with good information.

  46. Take a deep breath, Tinder Plus has finally arrived on your desktop web browser.

  47. Great post! Thanks for sharing the info

  48. Are you excited for NBA 2K18 Game? NBA 2k18 Game will feature NBA basketball games with real effects and customized players and teams. This game created with latest designed, game commentary, halftime shows and even various visual features and lighting, player animations and camera angles will be improved than previous series.

  49. pioneerseo says:

    Yes, extraordinary US Military drive. Likewise, in his post you have allowed to tune in about US Military. I truly value your work. A debt of gratitude is in order for sharing it.affordable hair loss surgery costs

  50. Thanks a lot for sharing this great resource. You always keep us updated with the latest information from the world of technology, and hence I visit your site so often. Keep up the good work and you can also check out all the latest info about iOS 11 features which is going to launch this year.

  51. Watch Movies says:

    I still don’t know about A KPI leads to action! 🙁

  52. This game created with latest designed, game commentary, halftime shows and even various visual features and lighting, player animations and camera angles will be improved than previous series.

  53. You want to play the best games and the most attractive, go to our website, it will not disappoint you.

  54. This topic has always been one of my favorite subjects to read about. I have found your post to be very rousing and full of good information. I will check your other articles shortly.

  55. Thanks for sharing the info

Post a new comment