Evaluating Training EffectivenessEvaluating Training EffectivenessIs the Knowledge Transfer really happening?
Evaluating the effectiveness of training on employees and clients is an often overlooked process. Often, the evaluation ends with the student submitting a course summary ("smile sheet") before leaving on the last day of class. Now more than ever, a more thorough plan must be implemented by training managers, human resources professionals, and supervisors. Not only is training an expensive proposition, but employees are now attending class in non-traditional environments...often with unpredictable results! These factors drive the need further as we determine exactly which mode of delivery works best for all categories of training and education.
What we will cover...
After beginning with an overview of evaluation, we will concentrate on the full scope of activities necessary to properly evaluate training. Since not all training efforts are the same, we will compare and work with the variety of tools and processes necessary to do so. Shown below, as course units, these include:Upon completion of the workshop, the attendee will be able to conduct an evaluation of a course using the reference book as a resource. Evidence of success will be completion of exercises for the various components of an evaluation.
Unit 1 - Client Negotiation
Unit 1 will begin the course with defining client needs. This unit will also introduce the first of many exercises and case studies. Topics include:
Exercise - Using roll-play involving another participant, this exercise will allow the attendee to practice gathering preliminary data necessary to conduct an evaluation of a training program.
Unit 2 - Considerations for Planning an Evaluation
Data gathering, with a keen eye on resources and constraints, is vital to effectively planning an evaluation. This unit will identify these components and offer an opportunity to test knowledge with two post-unit exercises. Some of the components include
Exercise 1 - A new course has been questioned by management. How have the course attendees' performance improved since attending the course?
Exercise 2 - A once popular course has been receiving numerous complaints. Your mission is to find out why, using the data gathering ingredients you've just learned.
Exercise 3 - A previously effective course has recently been receiving a sizable number of complaints. Strangely, there doesn't seem to be a clear, identifiable reason for the complaints. Your mission is to "repair" the course after relying upon the resources you have learned.
Unit 3 - Questionnaires
Questionnaires represent crucial data gathering mechanism. Yet, despite their apparent simplicity, many questionnaires are less than effective in providing information to the client. In this unit, we will explore the typical problems and solutions for "weak" questionnaires, and how to package them to maximize results!
Exercise - A client requires you to evaluate an existing seminar to determine if it is effective in its current format, or if it requires major maintenance. Naturally, you'll need to carefully author your questions to solicit precise responses.
Unit 4 - Debriefing a Group
Ideal for quick evaluations of training programs, a Group Debrief is a directed discussion which occurs when a small group of people, usually no more than fifteen, are asked by a facilitator to rate a training program and respond to specific questions about it. This unit will discuss:
Exercise - Attendees will participate in a group debrief session using our Case Study course.
Unit 5 - Interviews and Focus Groups
Since an interview is a personal, live contact with a respondent, it can provide the opportunity to hear directly about a subject. Today, interviews can be conducted face to face in person, by telephone, or over a video teleconference network. Focus groups involve six to twelve participants who gather to address a one or two specific issues.
Exercise - In order to gain information on our Case Study course that some instructors feel requires "significant maintenance" you decide to organize a focus group. This effort will call upon your ability to work within the guidelines as they apply to the Case Study situation.
Unit 6 - Observations and Subject Matter Expert Reviews
An observation is primary evaluation technique conducted by one or more persons. We will focus on when and how to effectively structure observation "content" and placement.
Unit 7 - Written Plan and Client Concurrence
This unit will center on the correspondence between client and evaluator necessary to obtain an understanding of the evaluation's objectives and scope. We will examine actual documents from previous efforts in evaluation.
Exercise - Using our Case Study course and notes gathered from previous exercises as input, you will write a detailed plan to submit to the course manager.
Unit 8 - Administration of Data Collection
Data gathering requires the adherence to specific guidelines that include developing and assembling of data collection materials, testing and revising those materials and planing specific work assignments. This unit will review these processes, as well as:
Unit 9 - Data Analysis
Once the findings are in, it's time to analyze the data. This unit will focus on organizing the compiled data and data interpretation. From this point we will draw conclusions and make recommendations. As always, we will include written examples of actual data analysis documents.
Unit 10 - Findings and Recommendations
Evaluating the findings will lead to recommendations to the client. Before this is can be communicated, the "cause and effect" relationships must be clearly understood. This section will focus on the guidelines for making recommendations and include a practice exercise
Exercise - Attendees will participate in a practice exercise designed to evaluate a new, vendor-provided training program on project. You will need to review the findings and prepare a recommendation to the training director.
Unit 11 - Results Reporting
An evaluation report should communicate the findings to others by providing a record of what was done and what was discovered. We will examine the contents of a properly written report to the client. Again, all attendees will receive samples of actual reports that can be modified for specific training evaluations.
Most attendees should have had supervisory or managerial experience in training and education. As such, they may also have served as instructors and have completed undergraduate or graduate coursework in education or instructional technology.Either of these experiences should allow the attendee to benefit from this course.
|
Return to AR&D Homepage |Online Data Communications GlossaryContact us
Copyright ©1999 Ara Research & Development.
All Rights Reserved. AR&D liability, trademark and document use rules apply. Your interactions with this site are in accordance with our privacy statement.