~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Get Zeroed-In on Learning Measurement
Issue 7
Fall 2007
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In this issue ...
-- Talent Measurement: A management necessityGreetings from Zeroed-In Technologies!
Get Zeroed-In on Learning Measurement is a quarterly newsletter devoted to learning and performance measurement and the people and processes surrounding it. Each issue contains feature articles, benchmarks, case studies, opinions, and upcoming events relating to learning measurement, learning analytics, and strategic reporting inside and outside the learning organization.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Talent Measurement: A management necessity
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The phrase "managing talent" garners much more interest and support from the executive suite than traditional references to the same set of processes years ago. "Managing learning" just doesn't seem to have the same pizzazz. As a result, executives are keen to know where they align in terms of attracting, building, engaging, leveraging, and retaining top talent. The opportunity for talent managers is to take these processes and develop a series of talent measures that best define success in the eyes of their stakeholders.
The best talent measures are directly influenced by the underlying work process and outcomes of your talent initiatives. The purpose in managing any process is twofold. First, you aim to improve the process and second, you aim to produce some desired outcome. Improvements are a direct result of executing the process efficiently and effectively. The outcomes are the performance outputs of the process.
With these aims in mind, your talent measures can be classified into two categories: execution measures and performance measures. Execution measures relate to your internal organizational processes that drive the talent management initiative. Performance measures relate to the outcomes and effects those processes have on your audience (i.e. the talent) and your organization (i.e. the stakeholder). Below is a sampling of measures for one of the key processes, attracting talent.
Execution measures for attracting talent include:
Performance measures for attracting talent include:
By setting targets for your talent measures and collecting the measure on a recurring basis, you now have the foundation for producing a composite-level scorecard for each talent goal and indicator. Score cards commonly display progress in terms of stoplight-style images (e.g. red, yellow and green). These color-coded displays are a direct reflection of the proximity that the actual measurement value has at a point in time to the target that you set for that period.
Just defining and collecting measures won't help you manage any better than before, unless you use the measures as a basis for effective decision making. As you prepare a set of candidate measures, it's important to put your measures through a simple test: If the measure goes up, what will you do? And conversely, if the measure goes down, what will you do?
This test will help you formulate a decision-making strategy for each measure and validate whether the measure really matters. You don't want to select measures that are easy to explain away, although some managers like to choose "soft" measures because they limit the level of accountability. Unfortunately, measurement alone doesn't change behavior, so tying in accountability is crucial if you want the process to improve.
Read the full article here …
http://www.talentmgt.com/index.php?pt=a&aid=241&start=6157
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Benchmarks: Alignment of training to organizational strategy
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Recently, Chief Learning Officer Magazine published its 2007 Chief Learning Officer Business Intelligence Industry Report. Here are some highlights from the Executive Summary as it relates to alignment.
Of the surveyed audience (1322 respondents), only 17% report excellent alignment of training to organizational strategy. In other words, only 17% believe the Chief Learning Officer’s goals are directly reflecting company objectives.
The three biggest challenges in establishing or maintaining a link between learning and business objectives are: demonstrating learning’s impact on strategy; identifying, defining, and implementing strategy; and getting executive level buy-in.
On average, more than half (52%) of respondents indicated that they are not satisfied with the level of alignment effort expended to link the learning strategy with business objectives.
Get the complete Executive Summary here …
http://www.clomedia.com/biireport/BZ_2007_Summary_r2.pdf
Visit our website at www.getzeroedin.com to take our latest poll “What is your primary learning measurement & reporting need?” We’ll include the results in our next newsletter.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Featured Measure: Learning alignment indicator
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
What it means: The learning function has defined its strategy and prioritized its initiatives in line with the organization goals and stakeholder needs. A channel of communication is present and utilized to facilitate incoming requests, changes, expectations, resource allocation, and progress with stakeholders.
How it's measured: Qualitative and/or quantitative value ratings on key programs by managers; planning meetings scheduled –vs- held with key stakeholders; percent of strategic learning initiatives agreed to and signed off by stakeholders
How it’s used: The learning alignment indicator demonstrates to stakeholders that the learning function is delivering strategic value from the investments being made. Additionally, this indicator depicts the maturity of the learning organization’s planning and prioritization process, indicating when and where improvements are needed.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Research: LMSs continue trend of data rich, information poor
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If you needed more evidence that those who manage learning are unhappy with their ability to measure and report on their programs, new research finds more organizations now using workarounds for reporting learning metrics than are using LMS-generated reports. A market study conducted by Expertus and TrainingOutsourcing.com, received over 300 responses and found that those using internally developed software or Excel spreadsheets or manually compiled reports now outnumber those using the standard or custom reporting capabilities of their LMSs.
Other study highlights include:
+ 81% had a high level of dissatisfaction with the reporting capabilities of their LMS
+ 47% said custom reporting requirements were too onerous
+ 46% said overall reports functionality was too complicated to use
Gaps between the actual reporting capabilities of LMSs and the expectations of LMS buyers exist because salespeople often oversell LMS functionality and minimize the customization requirements and related technology challenges. Also, LMS buyers often don’t ask detailed questions about reporting capability that would uncover potential shortcomings.
Study results were reported in the October issue of TRAINING magazine. For the full article visit...
http://www.trainingmag.com/msg/content_display/publications/e3i79194959d978bedee932d5b37971e44d
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Best Practices: Managing your learning project portfolio
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Recent trends in measurement and reporting show an increase focus on organizing and monitoring activities of the learning project portfolio. If you’re not familiar with the concept of the learning project portfolio, it’s simply the collection of all the learning projects that the learning and development (L&D) organization is working – past, present, and future. If you consider how L&D functions, it’s essentially a project-based organization, continually queuing up projects and churning out deliverables for its internal and external stakeholders. Success of the organization lies in the leadership’s abilities to prioritize, manage, and deliver on their project goals and expectations. The key measures of success are commonly adherence to budget, on time delivery, progress to date, and resource utilization. A growing number of organizations also measure customer satisfaction as it relates to the project management process and final deliverables. Incorporating concepts from Six Sigma (e.g. Voice of the Customer) prove essential in establishing project success criteria and the expectations on which to measure satisfaction.
More over, learning managers seek to answer questions presented by their peers “What is learning working on today? What did you do last year? And what do you have coming up on your schedule?” Without a centralized place to manage and track your learning project portfolio these types of questions become increasingly difficult to answer with accuracy. Doesn’t your learning management system (LMS) track this information? Well … no, it doesn’t. The LMS is a post-development infrastructure designed to keep track of what training is available, who needs it, whose had it, and when. The LMS works with learning projects after they are completed, not through planning, design, development, and delivery For e-learning projects, the learning content management system (LCMS) environment may be a little closer to project management, but even the LCMS lacks capabilities to manage budgets, resource allocation, vendor partnerships, and the like. Furthermore, adoption of a centralized LCMS infrastructure has been slow in the marketplace.
How about Microsoft Project? Well, sure, the project itself can be managed in MS Project and it’s a great tool for the project manager (PM). But how do you aggregate statistics for your entire project portfolio off the desktops and laptops of your PMs? It becomes very tedious without a centralized inventory.
Lately, thought-leading organizations have embraced learning measurement and governance tools like CLO Dashboard(tm) to manage, track and measure their learning project portfolios. These tools provide learning organizations with a centralized place to charter and manage the project management process and deliverables from start to end. They also provide visibility and access to all of the key players, giving them an opportunity to submit online project status reports and open a dialogue with peers when areas of discussion or attention are required. As a result, leadership can monitor project success criteria, identify risks, and mitigate them to keep projects moving smoothly.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Pertinent Articles and Resources ...
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
• Previous Get Zeroed-In on Learning Measurement newsletters
• Seven key learning indicators your CEO really needs to know
• Learning from Key Learning Indicators
• Talent Measurement: A Management Necessity
• Measuring Success: Capturing the right metrics
• Measuring effectiveness with learning analytics
• CLO Dashboard - Performance scorecard and dashboard for learning
Additional resources available at: http://www.getzeroedin.com/resources.php
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Upcoming Events ...
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
• November 6-7, 2007 – Managed Learning Services 2007 -- San Antonio, TX
• December 11-12, 2008 - Advanced Learning Technology (ALT) Summit -- Cary, NC
• April 22-24, 2008 - Impact: The Business of Talent -- St. Petersburg, FL
Details about these events available at: http://www.getzeroedin.com/events.php
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Contact: Chris Moore, Zeroed-In Technologies
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
email: cmoore@getzeroedin.com
phone: 410.242.6611
web: http://www.getzeroedin.com
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~