On a video conference. Present are:
- OPS – Operations Manager
- HR – Human Relations Manager
- PM – Performance Manager
- LT – Learning and Talent Development Coordinator
- TC – Training Coordinator
- HG – Consultant (Hired Gun)
HR HG, I haven’t seen any quizzes or multiple choice tests (MCT) in your materials. Is that on purpose? Will we get to that later?
HG First let me ask you a question. What is your experience with MCT? Do you do well or poorly?
HR I love MCT. I know the correct answer is there if I can just recognize it.
HG You say, “recognize.”
HR Sure. I don’t have to know the answer, just pick out the one that looks familiar. It is so easy.
I figured out early on that I don’t’ have to “know” anything, just pick the answer that I seem to recall.
HG How have you done.
HR Wonderful. I always do well on MCT.
HG Anyone else want to comment?
PM Yeah. I hate MCT. I barely got through school with these tests. I am really glad they only wanted 70% as a passing grade.
TC I have a friend who would agree with HR, call them Friend 1. Friend 1 did poorly in High School, and then flunked out of college. Friend 1 took an entry level job with a large firm at little more that minimum wage.
However, Friend 1 was hard working and was recognized as having a lot of potential.
Are you all familiar with the College Level Exam Program – CLEP?
OPS Isn’t that something like the GED for high school diplomas?
TC Yes, only for college credits, up to 30 semester credits.
So, the firm offered Friend 1 a chance to move into the management track as long as they could score at least 30 percentile on each of the 5 exams.
LT Remind me again about percentile. I think I know but maybe not.
HR Percentile relates to a position in a group and is often shown as a bell curve. If a person scores better than 40 percent of the entire group, that person is at the 40th percentile.
As I recall, 30 percentile is considered a passing score for someone with two years of college.
TC Yes, and 50 percentile is considered passing for a college graduate.
Friend 1 agreed to take the exams, but was really concerned about the math exam. Friend 1 had barely passed basic math courses in high school and had never taken any advanced math. They even arranged for tutoring if they failed the math exam.
PM How did it go?
TC Friend 1 scored around 90 percentile on four of the exams and 51 percentile on the math exam. Friend 1 told me they have no idea how they did that, but they got the promotion, and went on to a successful career in management.
PM Well, I have a different story. I have a friend, call them Friend 2 who has the same trouble with MCT as I do.
Friend 2 went into medicine. After undergrad and professional college, they barely passed their board exams to get their license.
By the way, do you all know that your doctor only has to get a 70% on their boards to pass?
OPS 70%? What about the other 30%?
PM That is just the way of the world.
Anyway, after a few years Friend 2 did a residency and received a masters degree in a specialty. Now they have struggled for seven years but are unable to pass the specialty boards. They freeze up on MCTs.
LT How does Friend 2 get a job.
PM Well, their masters degree and their obvious expertise during interviews assure employers that they really do know their stuff, and that the board exam doesn’t matter.
Also, Friend 2 has been around long enough that their experience is worth more than their education.
HG So, we have two extreme but true stories that illustrate the fallacies behind MCT.
Bear in mind also that those tests are written by people who have advanced degrees in MCT.
TC That doesn’t give the rest of us much chance, does it?
HG Now, let’s go back to the earlier question about the 70/30% passing business.
I have a hypothetical situation. Think about the written driver’s test we all have to take to get a license to drive. The passing score is 70%.
Consider Driver 1, who takes the written test and gets a passing score and a license. One of the questions missed, however, is the one about an octagonal red sign with large white letters “S-T-O-P.” Driver 1 misses that question.
A few days later Driver 1 gets a ticket for running a STOP sign. In court Driver 1 pleads not guilty on the basis that is was already proven that Driver 1 did not know what a STOP sign was and therefore is not liable.
HR Come on! That is ridiculous!
HG Of course, but back to your doctor. Are you comfortable that they pass with 70%? Does the other 30% matter.
OPS Ah! Now I understand something I didn’t really think about before. I have met doctors that told me they only studied certain things knowing they had a 30% fudge factor.
I guess we really do let the examinee decide what 30% they don’t need.
LT But can MCT be used to reinforce learning or understanding?
HG Well, think about it. In the case of Friend 1 you reinforce the attitude of being really smart. In the case of Friend 2 you reinforce the attitude of being really stupid.
How can either of those attitudes be healthy?
HR So I guess we have answered our own question. We don’t use MCT because they don’t give us much confidence that the results mean anything.
HG Yes! Everyone in my profession knows that even though they try to ignore it, even all the elearners.
PM Yeah. I have been reading all kinds of things, and sooner or later they all admit that the only way to assess T&D, L&D, whatever, is skills application and demonstration, not quizzes or MCTs.
HR Even soft skills?
HG Yes. The assessment is through peer agreement that they have demonstrated the required skills. We have demonstration, peer assessment, coaching, and encouragement. It is all very interactive and a lot of fun.
LT Then why do people waste so much time on MCT?
HG In my opinion? Sheer laziness. MCT are easy to grade. Meaningless but easy to grade.
Please register with us and add your comments here on the blog site.
If you have personal comments or questions for us, you may email us at firstname.lastname@example.org.
We also invite you to join our conversations related to these blogs, our Facebook group, and our LinkedIn group shown below.
Fred Parker is the Director of Course Design and Development at https://www.PathfinderCoaches.com. Fred has half a century as a Performance Management Consultant designing, developing, and delivering performance-based training including all manner of technical training, individual development soft skills, and basic leadership. Clients include all sizes from a local sandwich shop to the military to fortune 500 multinationals. Now Fred is converting previous ILT courses to remote delivery courses available on our web site.