Contract Training Edge

The Level 1 Evaluation

If your college is like most across the country, you have a standard Level 1 Evaluation form you ask participants to fill out at the completion of all open-enrollment and contract courses. This form is designed to collect information from our customers about their reaction to the learning event. Most of these Level 1 Evaluations are mostly about us and our environment. Do your questions look something like this?

"The facility was…"
"The instructor was…"
"The materials were…"
"Which of the topics covered were most/least valuable…"

We ask our customers their thoughts about our content, our instructor, our methods, and our environment. Instead, we should be asking them about them and how they are experiencing the learning event in relation to their needs.

I recently delivered a few workshops for college leaders across the state of California at the Statewide Contract Education Summit in San Jose, CA. During some down time I had the opportunity to sit in a session called "Measuring (& Driving) the Value of Training" delivered by Bruce Winner, Custom Training Manager at Los Rios Community College District (Sacramento, CA). Bruce’s presentation was as valuable as his name! While I was already very familiar with the Kirkpatrick Levels of Evaluation, Bruce introduced an approach to collecting participant feedback that touched on all four levels (Reaction, Learning, Application, & Impact) in the context of a Level 1 Evaluation. Needless to say, I left the session feeling this was a better way to capture Level 1 feedback and couldn’t wait to get back to my office to use this "blended approach" to make improvements to the Level 1 Evaluation I have been using. With Bruce’s approval, I’m happy to share some of the major components of this new and improved Level 1 Evaluation. Some language may need to be modified to support different program areas.

REACTION (5=strongly agree, 1=strongly disagree)

I found the knowledge, skills, tools, and techniques from the workshop to be relevant to my job.
My understanding of the workshop material was increased by the instructor’s use of lecture, exercises, and discussions.
I found that the participant guide, handout materials, and job aids used in the workshop aided my understanding of the content presented.
My comments related to course relevance, instructor, or materials: (open ended)

LEARNING (5=extensive knowledge, 1=very little knowledge)

My level of knowledge regarding __________ prior to this workshop was…
My level of knowledge regarding __________ after completion of this workshop is…

APPLICATION (5=strongly agree, 1=strongly disagree)

I will be able to apply the knowledge and skills learned in this workshop when I return to my job.
I estimate I will use the tools or techniques learned in this workshop to the following extent (circle one): None, Limited, Moderate, Significant, Very Significant
These are the barriers that I anticipate may hinder my ability to apply the new knowledge/skills gained from this workshop: (open ended)

IMPACT

I think these impact measures will result from the application of knowledge/skills from this workshop (circle all that apply). Increased … Quality, Revenue, Margin, Productivity, Efficiency WORKSHOP SUGGESTIONS

Keep this! I found this useful and relevant: (open ended)
Drop this! I found this neither useful nor relevant: (open ended)
Other suggestions I have regarding this workshop: (open ended)

SATISFACTION (5=strongly agree, 1=strongly disagree)

Overall, my learning needs in this subject area were met by this workshop.
I would recommend this workshop to others.

Here are a few reasons why this approach to a Level 1 Evaluation may work better for your college:

If you didn’t define specific Level 2, 3, or 4 measurements on the front end of the learning initiative, you now have some data about your clients’ expectations in these areas.
You are demonstrating to your clients that higher levels of measurement are important to you.
You will collect data to support a “Client Report Out” and/or future conversations about new opportunities.
Questions are asked from the learner perspective (not the trainer perspective).
Collecting more “application” and “impact” data from your client will support a longer term, strategic partnership with them.

I’m not suggesting this evaluation should take the place of a thorough Level 2, 3, or 4 evaluation, but it is a way to capture more useful data that a traditional Level 1 Evaluation will not provide. If you would like to explore how Contract Training Edge can help your team improve business development skills and develop your own team dashboard, please contact Greg Surtman at (216) 509-6398 or gsurtman@ctedge.net.