Worse Than Useless: How multiple choice tests can negatively impact learning, and what to do about it

ExamCritics of standardized testing have long held that multiple choice tests are useless for evaluating a student’s real understanding of material. They wrongly assume that the testing itself is learning neutral- that taking the test has no impact on students, aside from wasting their time.

Policy makers who support standardized testing also view test taking as learning neutral, despite long standing and robust evidence that test-taking has an important impact on retention of knowledge.

Generally, testing is an excellent way of increasing retention. But there is one exception: the multiple choice or true/false test without detailed feedback- a.k.a. the standardized test.

Most multiple choice questions contain at least one lure- a wrong answer that seems plausible or even likely to be correct. Some exams, such as the SAT, may contain multiple lures per question.

Let’s take an example:

An ovoviviparous animal is an animal that:

  • A. uses powerful scent glands to deter predators when threatened.
  • B. reproduces by producing eggs which are retained inside the body until ready to hatch.
  • C.  reproduces through laying eggs.
  • D.  consumes eggs as a main source of calories in its diet.

As you read through this example, your inner dialogue might sound like this:

A. Scent glands? No, that’s definitely not the right answer. B…Oh eggs! Yes, ovi means egg, that answer sounds right to me. Now for C… wait, this answer has eggs in it too. Now I’m not so sure if B is the correct response. D has eggs, too! But I’m pretty sure the term is related to reproduction, so it must be B or C.

Depending on what test strategy you have been taught, you may want to go with your gut and choose B, or go with C, which is statistically the most likely response on a multiple choice test.

(For your sake, I hope you chose B.)

From the journal Trends in Cognitive Science,

“The need for feedback is critical after any type of test, but it is particularly important for recognition tests (e.g. multiple choice, true/false, etc.) because test-takers are exposed to incorrect information. For example, on multiple- choice tests, students must identify the correct answer from a number of possible alternative answers (i.e. lures), most of which are plausible but incorrect. The danger is that because students learn from tests, taking a multiple choice test might cause them to learn incorrect information and believe that it is true. Indeed, recent research has shown that when students select a lure in a multiple-choice test, they often reproduce that incorrect information in a later test. (1)”

Why?

Memories can be thought of as traces in the brain. The more the thought is retrieved, the more embedded the trace becomes,  like walking a forest path over and over makes it easier to follow. When a student with a “faint” trace is confronted with a “lure” in a multiple choice test, he may take the bait, thus adding a new faint trace of learning to his memory.  Since testing greatly increases retention, this false trace will be remembered, producing wrong answers on subsequent tests. This result is true even of purely evaluative tests, such as the SAT. (2,3,4)

What You Can Do

The good news is that the negative effect of multiple choice and true/false tests can be completely eliminated through feedback. (2) If the student receives corrections for errors, the risk of repeating errors in future tests is eliminated.  The testing has the same positive effect as other test enhanced learning methods.

Application in the Homeschool

Do not use multiple choice tests, either as a method of evaluation or as part of retrieval practice, unless you can guarantee feedback for wrong answers. As much as possible, retrieval practice and assessment should require the student to produce a response from memory, rather than more passively choosing a response from several options.  The advantage of the homeschool environment makes this feasible, as most parents are working with a small number of students, making the load of test correcting lighter.

If a multiple choice test is unavoidable (such as required standardized testing), provide feedback and correction for all wrong answers to remove false traces. (Check your test’s instructions for legality beforehand).

 

 

(1) Roediger, Henry L. et al. (2010) The critical role of retrieval practice in long-term retention. Trends in Cognitive Sciences, Volume 15, Issue 1, 20-27.

(2) Marsh, E.J. et al. (2007) The memorial consequences of multiple-choice testing. Psychonom. Bull. Rev. 14, 194–199

(3) Roediger, H.L., III and Marsh, E.J. (2005) The positive and negative consequences of multiple-choice testing. J. Exp. Psychol. Learn. Mem. Cogn. 31, 1155–1159

(4) Butler, A.C. et al. (2006) When additional multiple-choice lures aid versus hinder later memory. Appl. Cogn. Psychol. 20, 941–956

 

One thought on “Worse Than Useless: How multiple choice tests can negatively impact learning, and what to do about it”

  1. Fascinating. I hadn’t thought about the idea that seeing (and especially choosing) the wrong answer on a multiple choice would reinforce the mistake. But I still remember a college chemistry test (not multiple choice) in which a faulty calculation caused me to get an answer I later learned was wrong. I think of the wrong answer each winter, mostly because I was so taken with the conclusion that urea was better than salt for deicing sidewalks. The image of sending little boys out to pee on the driveway just won’t go away.

    As I student, I loved multiple choice tests, especially standardized tests — for all the wrong reasons, of course. I’m one of those who are good at the kind of guessing required, including knowing what kind of thinking the test requires. I didn’t reject answer A immediately, on the grounds that maybe there’s a connection between the scent glands and egg laying that I don’t remember, but the other answers told me that’s not the direction the test was going. D I knew was wrong. I had to think between B and C, but the connection of “viv” with life did the trick.

    I briefly considered taking an extra SAT exam in a subject I had never studied, just to stretch my test-taking skills, but eventually decided I didn’t want to waste the time. Sadly, I’m not nearly as good at fill-in-the-blank tests.

Leave a Reply

Your email address will not be published. Required fields are marked *