Metrics, Metrics, Metrics! Measuring Digital Learning In Your Company
Not measuring and not reporting your value add/ business contribution is going to damn your learning organisation to ever deceasing budgets
What are your metrics to measure success when it comes to digital learning? Are you even making a difference with your digital learning?
It’s frankly depressing that when you ask learning technologists to talk about the metrics around what they do their natural inclination is to talk about the numbers of completions of their courses, not about the tangible value they have contributed to their business.
It’s as if, in the spirit of the Olympic ideal, it’s the turning up, rather than the winning that counts.
The harsh truth for most organisations is that actually, it’s the scientific and focused investment in winning that secures success. Measuring success is central to this, and those that do it well get the most funding and that builds more success.
But to be fair, it’s not just the learning technologists that suffer an inability to measure impact, it’s a malaise that affects L&D, talent management and HR as a whole. All too often the focus is efficiency and cost saving rather than the value added.
My personal view is that Kirkpatrick and Phillips are to blame for creating a groupthink that evaluation of learning is to do with to do with levels and a false assumption that measuring return on investment (ROI) is too hard or impossible to prove. The thinking almost seems to be, ‘We can never prove if learning is the reason for higher performance’.
Strangely I agree. Cause and effect can be incredibly complex. But here’s the rub:
THAT SHOULDN’T STOP US SETTING MEASURES OF SUCCESS FOR OUR LEARNING PROJECTS AND TO SEE IF WE HAVE MADE A TANGIBLE DIFFERENCE!!! And a failure to set these out in the design of any learning programme is just plain lazy!
If you live under the yoke of any CFO worth their salt – not measuring and not reporting your value add/ business contribution is going to damn your learning organisation to ever decreasing budgets!
My suggestion is that you look to operate the 80:20 rule and make sure that 80% of your programmes have demonstrable measures of success. This is essential if you want to be seen as a valued partner by your stakeholders and be able to argue for the optimum funding for your team. Sadly, from our experience at Fosway, we’d be lucky to see even 20% of learning measured in any form beyond who turned up!
So what can you do?
The more radical side of me would suggest relegating your learning evaluation programmes to the background and start implementing learning impact programmes that measure:
- strategic business success
- business unit performance
- development in personal capability – for your learning initiatives
This involves benchmarking where you are today and then measuring where people are after they have participated in their learning opportunities. Both in terms of their confidence and competence, but more importantly through demonstrable impact. And this means using a wider set of measures than those in an end of the course-learning assessment.
- Measuring and analysing performance ratings
- Getting 180 and 360 feedback from peers, managers and customers
- Pulling in quality assurance process feedback
- Using real business performance measures
- Harnessing your learning design to define what outcomes are needed
- Using multiple sources to show learning has made a difference ideally with an eye to business KPIs
With the advent of Big Data and machine intelligence, this would appear to be easier than ever to achieve as the technology is more ready to support these kinds of initiatives.
Interestingly, there are some major changes happening in provider space that go beyond the technology. There is a shift in how some are operating and it’s a change I expect to grow. It’s called ‘Success-as-a-Service’ which is challenging what we buy when we purchase the software. In this model, it’s not about buying a piece of kit or software – you’re buying the outcomes. There is “skin in the game” from the supplier and a clear focus for buyers. It’s a veritable win-win.
This approach has the potential to be very disruptive and fragment many existing software relationships. The challenge for learning professionals is that if you’re not able to articulate your value add – you will be very low hanging for external providers offering business success proposition.
So, whilst it may be tempting to simply react to the demands of your business and develop that learning programme; if you want to have a stronger chance of survival you’ll need to hold your customer down and agree on some measures of success. Sometimes kicking and screaming!
Because the simple truth is this: if someone offers them success rather than solutions – they will be quick to drop you and start buying. And who would blame them?
Not measuring and not reporting your value add/business contribution is going to damn your learning organisation to ever decreasing budgets.