Saturday, July 26, 2008

Answers In Mind

Adam CarleAdam Carle

Department of Psychology

Do you believe in common knowledge? I’m beginning to feel that I don’t. It seems a wealth of common knowledge about test taking exists, but the educational psychologist in me knows that too little research has empirically examined much of the common lore. As we enter the final week of summer session, I’m sure a few of our students have tests on their mind. As they study and prepare, many of our students ask us for advice.

“How should I study for the final?” (Don’t cram, space your studying)

“How many questions will it have?” (Enough to reliably and validly measure your achievement)

“Do you have specific advice on how to take the final?” (?)

Common testing lore (among students and professors) would have you answer that last one something like, “Don’t change your answer on multiple choice tests. Stick with your initial impression.” But, recently, some investigators decided to put this advice to the test. It turns out that this is ill founded advice. In a series of elegant experiments, Kruger, Wirtz, and Miller (2005) showed that, on average, students who changed their answers on tests faired no worse than if they’d kept their initial answers. In fact, on average, switched answers (when students doubted their initial answer) lead to better scores on average. This goes against what many of us have learned and what many of us believe we’ve experienced.

How can this happen? Relatively simply. Kruger, et al. suggest (and go on to show) that students more easily remember times when they changed a correct answer to an incorrect answer. As a result, they overestimate the number of times this occurs. They often don’t notice when changing an answer led to a correct response. Why should they? Most of us attend more strongly to negative feedback. Students do too. Because students painfully feel switches to wrong answers, students quickly and easily remember their occurrence. They overestimate their prevalence; misestimate their own experience; and subsequently choose a poor test taking strategy.

What can we do about this? The obvious. We can start telling our students that, when they doubt their answer, they should change it. We can start by telling our students that their initial impression may be wrong. We can start giving our students empirically grounded advice. Unfortunately, Kruger, et al. also showed that students didn’t want to change their test taking strategy, even when professors explicitly suggested the better strategy (switch) and even when professors showed students evidence for switching’s effectiveness. Kruger, et al. suggest that, in the end, students find a life’s worth of misleading personal experience hard to conquer.

However, I’m left wondering. What if all their professors gave this advice, not just one? What if they gave it regularly, not just once? What might happen then? Let’s find out.

Kruger, J., Wirtz, D., & Miller, D. T. (2005). Counterfactual thinking and the first instinct fallacy. Journal of Personality and Social Psychology, 88, 725–735.

1 comment:

Tina said...

that is interesting. I think I'll change the way I tell my students about this--no wait maybe that is not the right way to respond... I think I won't change, maybe I should change.. no wait maybe I won't after all.

I would think it all depends upon how confident you are in knowing the motive for your changing of an answer. Did you misread the question the first time? or did you misapply a theory or principle? or did you have an error in judgment?

Tina Holland, Communication Studies