07/06/2020 by Val D'Anna
Hitting all the learning targets for a new course under development is not enough when you’re after bullseyes. So before being released, each new I-CAR course gets a ‘test drive’ with a pilot. 
Did the course deliver information relevant to the student’s job?  Was navigation in an online course intuitive? Did instructors find live course demonstrations simple to understand? These are questions our instructional designers ask themselves throughout the process of developing new courses, which can mount up to hundreds of development hours. In a pilot, these questions are directed to students who represent the intended audience segment. 
As many as 25 participants are invited to online and virtual course pilots. Fewer are generally invited for live courses, due to limitations in classroom size, and for Hands-On Skills Development (HOSD) classes, as class management and tool availability can be an issue. “It’s helpful to have some pilot participants with little experience in the course subject,” I-CAR’s Jamie Jacobs, supervisor, Instructional Design, says, “but we also like veteran technicians with experience in the subject to provide constructive feedback.” 
Jacobs explains, “I-CAR has always conducted pilots for new courses. It’s important to gauge that we hit the mark for the identified learning objectives.”
According to Jacobs, pilots help verify:
  • good course flow 
  • accuracy and relevance of the content, including videos and graphics
  • course length (i.e. does a 30-minute course actually take 30 minutes)
  • student engagement. In HOSD pilots, for example, “Do students find the props useful and a good representation of what they would encounter in the shop? Are our sample problems real world?” Jacobs says.
  • activities/demonstrations are clearly understood and properly presented by the live or HOSD instructor. “We watch for any stumbling blocks where the instructor looks lost in the material,” Jacobs says.
 For live, virtual and HOSD courses, participants are required to attend a pilot at its scheduled time. For online courses, participants are given two weeks to complete the pilot. The process concludes with a course survey and feedback report. Participants use a rating scale to judge the course on broad criteria like interest level and relevance to job. There are also questions specific to the course content like “this course answered my questions about advanced features/settings of the featured tool.” Those who complete the course and submit feedback receive credit for the course once it is released to the industry.
Most feedback received from pilots is positive. For example, in the pilot for the online course “Chief MultiMig 525 – GMA Welder Operation and Tuning,” an owner of an independent body shop commented: “…a great overview of the welder. It would be a great help to me if I was helping someone work through some problems with weld quality.” 
Participants also make constructive critiques of the courses they pilot. These might include navigation suggestions, requests for additional photos and recommendations to streamline the amount of information covered. This feedback is evaluated by the instructional designer and I-CAR subject matter experts to ensure it is accurate and includes all the necessary information to support the course objectives. Jacob says, “All feedback we receive is tremendously valuable, even if it is not added verbatim into the course. It lets us know if something is unclear or needs to be presented in a different manner.”
If additional data is wanted to quantify pilot results, an overall customer satisfaction score is calculated. When a pilot goes smoothly —  and the vast majority do — the new course will be ready for release within six weeks. This directly reflects on the experience and depth of knowledge of I-CAR’s subject matter experts and instructional designers who develop I-CAR’s curriculum. It’s a team that’s always shooting for bulleyes because their job is so critical —helping equip industry workers with knowledge required to perform complete, safe and quality repairs.
Seeking Qualified Volunteers for New Course Pilots
“I-CAR is always looking for candidates to participate in the pilot process,” Jacobs says. 
To be considered, click  “Suggestions and Recommendations” to bring up an online form where you:
  1. Enter your contact information. 
  2. In the “Idea or Suggestion” box, type “Pilot Participant” along with a brief summary of your experience and current role in the collision industry. “This helps us identify which future course pilots would be a good match for you,” Jacobs says.
  3. Finally, click the “Submit” button