Traditional use of the multiple-choice question rewards a student for guessing. This technique encourages rote memorization of questions to pass a lengthy exam, and does not promote comprehensive understanding or subject correlation. This begs the question; do we really want question memorizers to operate the machinery of our industrialized society? In an effort to identify guessing on answers during an exam within a safety-critical aviation pilot training course, a qualitative research study was undertaken that introduced a confidence-based element to the end-of-ground-school exam followed by flight simulator sessions. The research goals were twofold, to clearly identify correct guesses and also provide an evidence-based snapshot of aircraft systems knowledge to be used as a formative study aid for the remainder of the course. Pilot and instructor interviews were conducted to gather perceptions and opinions about the effectiveness of the confidence-based assessment tool. The finding of overall positive interview comments confirmed that the pilots and flight instructors successfully used the confidence-based assessments as intended to identify weak knowledge areas and as aids, or plans, for their remaining study time. The study found that if properly trained and administered—especially through a computer-based medium—a robust confidence-based assessment tool would be minimally-burdensome while offering worthwhile benefits.