~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Get Zeroed-In on Learning Measurement
Issue 4
Spring 2006
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In this issue ...
-- A vote for learning governance
-- Benchmarks: What they really tell us
-- Featured Measure: Learner attrition
-- Talent Management Summit 2006
-- Opinion Poll: Speed of delivery vs. time to competency
Greetings from Zeroed-In Technologies!
Get Zeroed-In on Learning Measurement is a quarterly newsletter devoted to learning and performance measurement and the people and processes surrounding it. Each issue contains feature articles, benchmarks, case studies, opinions, and upcoming events relating to learning measurement, learning analytics, and strategic reporting inside and outside the learning organization.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
A vote for learning governance
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Learning governance is a critical, yet frequently overlooked and under resourced, process in many organizations. Good learning governance is enabled by creating structure, processes, and values that enable good and actionable decisions that in turn lead the organization to some desired result. Strategic planning, execution, routine oversight and monitoring, measurement and reporting, problem resolution, and communication are the imperatives of a successful learning governance initiative.
Your learning technology infrastructure plays an important role in assembling and connecting the learning governance pieces. However, one area that is frequently relegated to a myriad of decentralized spreadsheets, documents, and other electronic or paper files is the process of learning portfolio and learning project management. Learning governance is dependent on this sub-process as a means of planning, oversight, monitoring, and reporting of your strategic learning initiatives, be they content development, technology deployment, or otherwise. As noted, project-related documents that house this information and other valuable aspects, like the allocation of human and physical resources, are spread across the organization’s desktops. Without a centralized repository to store information about past, current, and future learning initiatives, executives and managers have a difficult time ascertaining what the learning organization is currently working on and in what state its’ in, much less, what they did last year, and what they have upcoming on their agenda. Planning and decision making becomes more difficult and untimely without a complete and federated view of your strategic learning projects.
Learning management systems help manage and govern the processes related to learning content alignment, delivery, and tracking, but alone they do not provide structure for learning project management and the functions surrounding it. But it is the strategic learning projects that spawn much of the content and even the infrastructure that manages and delivers it. Governing these projects from a full lifecycle perspective (e.g. business justification, design, development, deployment, and impact analysis) is equally important in driving the key learning indicators of success for your organization as the content and infrastructure themselves.
Alignment of these projects is another concern. With limited funds to invest, learning organizations must ensure they target their resources into the right initiatives. Even the smallest work tasks can add up to large expenses over time. The alignment, monitoring, and governance of the work that your learning organization is doing is at the core of performance management. Organizations have found the performance management process valuable but tedious, particularly when trying to rollout an enterprise employee performance management (EPM) process. EPM is too granular and perhaps overkill for most organizations. Corporate performance management (CPM) brings it up a level (e.g. departments, teams, projects) and institutes structure and processes that favor learning governance. And, the cost to value ratio between CPM and EPM systems are much higher for CPM.
By combining CPM capability with strategic learning project management, learning executives and managers have a centralized view of how success is modeled for the organization, the strategy they must pursue to realize their goals, the measures that show progress, and the work their organization must do to influence those measures and thereby executing the strategy. With this valuable information in hand, key decision makers can govern the learning and development process and elect to invest in the strategic learning initiatives that will bring about the highest return for the business.
Whether it’s an individual decision or committee vote, an informed decision is always better. But remember, it’s not just in learning governance where your vote counts!
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Benchmarks: What they really tell us
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Many organizations look to benchmarks as a means to determine what types of information they should be collecting and measuring on a regular basis. It’s true that benchmarks can offer insight into a variety of common metrics, but they don’t necessarily tell organizations what are the right measures to track. Benchmarking is a very valuable and important practice, but benchmark measures may not naturally align to the measures of success for an organization. As an example, there are benchmarking efforts underway that measure and categorize the % of learning delivered by topic area (e.g. basic skills, customer service, leadership, technology) across various industries. The benchmark results can tell us where relative organizations are investing by way of learning content consumption and distribution across key topic areas. Unfortunately, the benchmark data rarely captures the profitability or success of the reporting organizations. So although we can see on average where our industry or similar size companies are investing, we don’t know what impact or outcomes those investments are driving. They simply become comparative numbers that tell us what our content consumption and distribution looks like relative to others. Should we invest more or less in certain topic areas? The benchmark results do not tell us the right answer. We need to make those decisions ourselves based on our organization’s strategic goals and the objective-level strategies that we will employ to meet those goals.
Benchmarks tell us where and how organizations are spending and investing their resources – financial, human, and physical. Benchmarking is a valuable and necessary process, and we encourage organizations to participate as much as possible. But don’t look to benchmarks to provide instant answers on what are the right measures of success for your organization. For those answers, reflect on your strategic goals, what work effort you need to employ to reach those goals, and what are the key measures of progress that plot your progress in accomplishing that work.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Featured Measure: Learner attrition
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
What it means: The fall out rate of learners in key programs, typically new hires that enroll and begin specific learning programs (e.g. orientation) but do not exit the on-boarding process normally or successfully.
How it's measured: # of people that exit the program(s) abnormally or unsuccessfully divided by the # of people that begin the program(s)
How it’s used: Learner attrition is used for staff capacity planning, as well as an expectation setting metric, when staff are required to successfully complete key programs prior to reporting for duty. As an example, if a business unit requires 12 new workers and learner attrition for the associated orientation program is 0.15 or 15% of the original enrolled participants then the business unit should enroll 15 candidates into the program to ensure that 12 successfully complete the program and report to work.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Talent Management Summit 2006
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Talent management is one of the most talked about human resource initiatives in the industry today. Simply put, talent management is the process of managing the supply and demand of talent to achieve optimal business performance. This effort requires a keen focus on talent strengths and weaknesses and a strong commitment to organizational goal alignment. If you are responsible for your talent management initiative, we cordially invite you and your colleagues to join us at the upcoming American Strategic Management Institute (ASMI) Talent Management Summit 2006 from July 24-26 in Boston, MA.
Talent Management Summit 2006 will provide comprehensive coverage of strategies and practices to enhance current talent management processes within your organization. "Best in class" practices for workforce analytics, succession planning, recruitment and retention, workforce planning, HRO, employee engagement, and development and training are all addressed in this program.
Zeroed-In Technologies president, Chris Moore, will be presenting "HR Measures: Tracking and Evaluating Your Workforce Assessment Success" on July, 24 at 11:00 AM and “Employee Approval: Measuring the Satisfaction of Your Workforce” on July 25, at 10:00am.
Receive a 15% discount on enrollment by using promotional code: W143/zero.
For more details and registration, visit http://www.asmiweb.com/Events/talent.html
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Opinion Poll: Speed of delivery vs. time to competency
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Recently a question was raised in a conversation in which a newcomer to the learning field was having, to wit: "Content can be delivered via asynchronous e-learning in less time than via classroom instruction" suggesting that 2x hours of classroom time can be delivered in x hours via asynchronous distance learning options. A follow up question was then posed, “Is there a general rule of thumb that has evolved from our experience with e-learning on its time measured efficiency in delivering information?”
It’s certainly an interesting point. I agree that asynchronous content can be delivered in less time than classroom because you have less interaction, distractions, etc from the audience. But an increase speed in content delivery does not necessarily increase the time to competency. Sometimes the chosen medium can have negative effects on the quality of content delivery. As a result, a learning activity that is badly designed or not conducive to e-learning or even classroom delivery could affect the retention and application of the knowledge. Not all content is conducive for cross-platform delivery. Of course, learning content vendors may argue with me there. But that’s half the fun of writing an opinion column.
As far as a given multiple (e.g. 2x, 1.5x) for time-measured efficiency, that’s certainly a good candidate to do some benchmarking with organizations that have converted specific ILT training to e-learning. There are a number of reasons that come to mind for why organizations convert ILT to e-learning, such as reducing travel costs, and increasing learner participation, and I’m sure that increasing time to competency is in that mix in some way or another. A comparison benchmark of this measure requires some historical tracking of relative impact, whether it’s real business results, or something like a job start date (e.g. “I’ve just been trained and I’m reporting for duty!”) By comparing the original durations of the ILT content with its impact against the new durations of the asynchronous content with its corresponding impact, a time-measured efficiency metric is possible.
I’m now inspired to do some benchmarking myself, at least in the form of polling. Below is a link to a new polling service now present on our website.
Poll question: “What is the average time measured efficiency rate of e-learning content over traditional classroom content?”
Submit your response at: http://www.getzeroedin.com#poll
You can only submit the poll once, but you can go back and view aggregate results. I look forward to seeing and sharing your responses. At a minimum, it will tell me how many of you actually read this newsletter from top to bottom!
- Chris Moore
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Recent News Items ...
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
• Zeroed-In releases CLO Dashboard v2.0 for learning governance
• Global Knowledge adds CLO Dashboard to powerhouse suite behind its Global Learning Platform
Additional news items available at: http://www.getzeroedin.com/press.php
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Pertinent Articles and Resources ...
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
• Previous Get Zeroed-In on Learning Measurement newsletters
• Seven key learning indicators your CEO really needs to know
• Measuring Success: Capturing the right metrics
• Measuring effectiveness with learning analytics
• Using models to manage strategic learning investments
• CLO Dashboard - Performance scorecard and dashboard for learning
Additional resources available at: http://www.getzeroedin.com/resources.php
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Upcoming Events ...
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
• June 11-14, 2006 - The 22nd Annual Training Directors Forum – Palm Springs, CA
• June 18-21, 2006 - 2006 IPMA-HR Eastern Region Conference – Washington, DC
“Human Capital at the Capitol – Achieving Monumental Results”
• July 24-25, 2006 – 2006 Talent Management Summit – Boston, MA
Details about these events available at: http://www.getzeroedin.com/events.php
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Contact: Chris Moore, Zeroed-In Technologies
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
email: cmoore@getzeroedin.com
phone: 410.242.6611
web: http://www.getzeroedin.com
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~