Over the past months we’ve listened intently to executives from a number of learning management system (LMS) companies discuss where they’re taking their next generation of learning and talent suites.  Last fall we participated in Saba’s annual global customer conference – Saba People 2008 – which attracted over 600 customers, partners and industry thought leaders from Europe, Asia, Latin and North America to Washington, D.C.  As with other LMS organizations, Saba’s major new product announcement at the conference was the pending release of Saba Social. 

Social networking platforms combine a rich personal profiling ability with competency frameworks to enable real-time collaboration and comprehensive networking among people with common interests. These tools enable organizations to engage employees, partners and customers more effectively, build connected corporate communities, and accelerate high-quality knowledge exchange and information flow to foster innovation and increase productivity.

As more organizations invest in social networking tools like Saba Social, they will want to measure the value that these tools add to organizational productivity. Management and vendors alike are betting on the fact that employees will use these tools in a similar manner that they do for their personal use, platforms such as FaceBook, MySpace, and Twitter.  If employees do invest workplace hours in using the company installed social tools, management will want to know how well one’s social content and interactions – their lessons learned posts, how-to videos, etc. - are rated by, recommended, and leveraged by their peers.  And most importantly, how do these tools create value.

At the Saba event, customer interest for measurement around social networking was apparent by the number of conversations we had about the suitability of our CLO and LMS Dashboard™ platforms. Our pre-built dashboard platforms serve to measure any and all learning and performance initiatives, and come without the significant consulting and service tail of the enterprise business intelligence (BI) solutions.  These points tend to resonate well with L&D executives wanting greater visibility to the impact an investment in new initiatives like social networking will have.

As always, let us know your thoughts on how measurement is improving what you do in your organization and where your thinking stands on social networking...

- C Moore

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Selling the 'C' suite on results
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Almost instinctively, executives discount reports on learning’s impact as too simplistic or attributable to non-learning factors. If those reports aren’t improving learning’s standing, you’re missing real opportunity.

This is the time of year when annual learning summary reports get compiled and presented to executives and other key learning and development (L&D) stakeholders. Often thick with numbers sprayed across graphs and tabular charts, the reports typically lay out totals for how many employees were trained, in what areas and the costs incurred for the year’s programs and initiatives. The better reports attempt to link learning results with the organization’s results. And it is here that most of these reports lose their audiences.

Too often, executives review the linkage portion of learning reports and discount or even dismiss what they read in favor of crediting a range of non-learning factors for the business gains: the new strategy or personnel, better salespeople, more favorable market conditions and so on.

To learn how you can improve reporting to your executives, read Zeroed-In's full article in the February issue of CLO Magazine. 

http://www.clomedia.com/features/2009/February/2533/index.php

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Introducing an answer to the question 'What should we measure?'
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Zeroed-In recently released the Impact Measurement Library™ to help organizations implement the advisory firm Bersin and Associates' new enterprise model of learning measurement. Most traditional learning measurement models ignore factors that are critical to a training organization’s success and exclude key operational measures important to senior executives and other stakeholders.

That's why, after five years of research, Bersin & Associates developed the Impact Measurement Framework®. The centerpiece of Josh Bersin's The Training Measurement Book, the framework highlights nine areas for measurement focus – satisfaction (reaction), learning, adoption, utility, efficiency, alignment, attainment of client objectives, individual performance, and organizational performance. The framework identifies key metrics in each of these areas that give organizations a true assessment of program effectiveness – and meaningful data to discuss with business unit and senior executives.

"Bersin & Associates' Impact Measurement Framework has clarified for McAfee Learning how to deliver measurable results," says George Brennan, CLO of McAfee, Inc., the leading dedicated security technology company. "Now we have the specific measures and an automated tool, CLO Dashboard, that gives us and our stakeholders on-demand visibility to the impact McAfee Learning enables."

Incidentally, McAfee’s Brennan has adopted CLO Dashboard for the second time. Brennan was previously CLO at NCR in Dayton, Ohio where he brought CLO Dashboard in almost five years ago to strategically measure the impact learning and talent management initiatives were having across all NCR business units.  The tool is widely used at NCR today.

Zeroed-In's Impact Measurement Library allows HR and L&D groups to quickly know what should be measured and immediately start using the measures to get better visibility to the impact they are having in their organizations. Each measure in the library can incorporate user-defined data dimensions, such as learner demographics, to more finely analyze and describe the measurement trends as displayed in CLO or LMS Dashboard.

"The Impact Measurement Library will greatly help training and HR organizations quickly understand the most powerful ways to measure corporate learning and development," says Josh Bersin, author of The Training Measurement Book and president of Bersin & Associates.

Read more about the new Impact Measurement Library at: 

 http://www.getzeroedin.com/pr09242008.php

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Learning measurement in Europe
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~  

In Germany, a large, globally operating pharmaceutical maker no longer attempts to measure the effectiveness of their sales representatives; such is the power of German trade unions to bar managements from using employee data for performance evaluation purposes. A senior learning manager at Royal Dutch Shell has never heard of Kirkpatrick’s four levels of learning evaluation, and is instead a devotee of Drs. Kaplan & Norton’s Balanced Scorecard work. According to our sources, these are two of many market differences in learning measurement between the US and Europe.

Measurement trends, like business trends generally, tend to move from the US to Europe, taking hold first in the UK and other countries that more readily embrace US business culture and language. Ireland, the UK, the Netherlands, Denmark, Sweden and Norway tend to more quickly adopt practices from the US, with Germany, France, Spain and others likely to more slowly embrace new American ideas.

The reality today is that throughout Europe, the accountability issue is becoming increasingly important.  Like its US counterpart, the days where the L&D department could just spend their learning budget without some sort of expected return are over.  Challenges that Europeans face with trade unions, the great diversity in languages and culture, and other barriers are increasingly stretching the boundaries of current measurement practices.  Organizations are revisiting their measurement strategies and embracing non-traditional models in an effort to navigate through these changing times.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Featured measure: Development cycle time 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

What it means:  Development cycle lag time is a measure of efficiency of your learning development and back office processes.

How it's measured:  Take the lag time between the date a learning need gets identified and the date the learning solution gets deployed.

How it's used:  Development cycle time can be used in a variety of ways.  First, the measure serves as a justification and comparison benchmark between internal development and external or outsourced development.  Second, the measure predicts how quickly your organization can respond to the needs of your customers.  A low development cycle time becomes a great sales tool for the learning organization as it reflects a highly efficient learning organization.  A high development cycle lag time clearly identifies there is room for improvement – perhaps targeting a need to outsource development if that task doesn’t seem to be your organization’s core competency. 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Opinion: When should Level 1's level out?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

Kirkpatrick Level 1 reaction surveys, affectionately called “Happy Sheets”, can be a valuable tool for refining newly launched or recently overhauled learning programs.  But do you really need to continue administering the same reaction survey for mature and stable learning programs, those programs that haven’t changed for months if not years?

In the early days of launching a program, standardized learner feedback is essential.  It’s a proven aid in the quality assurance aspects of learning development and delivery.  However, once a program becomes mature and stable the value of these standardized instruments diminishes significantly.  That’s not to say there shouldn’t be a mechanism to capture anecdotal feedback from learners.  It’s simply that the standardized questions with Likert scale responses tend to flat line and level out at or near the rating when the program reached its maturity point.

Now when I think of flat lining, the first thing that comes to mind is “what can we do to resuscitate?”  But do we really want to resuscitate something that may be best served just being retired?  Many, including myself, don’t see a point in having learners respond to generalized course surveys that no longer drive value.  In my opinion, it starts to become counterproductive. 

Ok, so some of you are saying, “but we can’t just stop administering our Level 1 evaluations!”  And that’s not what I’m entirely advocating.  Reaction surveys should be administered during the first six months to one year of a program.  But after that you need to think about weaning away from them as the program reaches stability and maturity.  In replace of the surveys, you should institute a review process between content owner and stakeholder to ensure that the instructional material stays fresh and current.  You can’t rely on the learner to tell you that.  They don’t know what they don’t know!

If you don’t buy in to the thought of ending those surveys, then perhaps we can meet half way.  At some point, it just doesn’t make sense to continue asking every learner to respond.  If you can randomize questions on a survey, clearly you can randomize who the recipients of the survey should be.  So rather than forcing every learner to click, click, click down the rows of a monotonous survey, pick one out of every fifty, or one hundred, or one thousand learners, and make them feel special.  Perhaps give them a gift for taking the time to provide their feedback.   And finally, ask them relevant questions now that the program is mature: questions like – do you think we really need this survey? 

And that’s my opinion …

C Moore

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Meet Zeroed-In At upcoming industry events
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Impact 2009 -- The Business of Talent

Bersin & Associate's Impact 2009 is an event with a guaranteed return on investment. Every aspect of this groundbreaking research conference supports the theme, The Business of Talent™. You’ll leave with specific ideas, best practices, and new contacts to help you improve the business impact of your learning and talent management programs.

For details and registration information, visit: 

http://impact2009.bersin.com/ 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Pertinent Articles and Resources ...
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 •  Previous Get Zeroed-In on Learning Measurement newsletters 
 •  Getting and keeping a seat at the table 
 •  Seven key learning indicators your CEO really needs to know 
 •  Metrics for marketing learning programs

 •  Talent Measurement: A Management Necessity 
 •  CLO Dashboard - Performance scorecard and dashboard for learning