Blog Post

How To Use Multiple Choice Testing As a Learning (not Assessment) Tool

How To Use Multiple Choice Testing As a Learning (not Assessment) Tool

For my proposed Coursera course on the History and Future of Higher Education--intended for high school and college teachers, students, parents, and students themselves--I am designing every segment not only to give history and analysis of the forms of education we have inherited from the Industrial Age that invented so many of them, I also plan to pass on concrete advice I've gleaned from the thousands of teachers I've met of how to learn with, through, around, and often despite the clunky apparatus of contemporary education.   I'll be giving advice on ways to be creative within structures that seemed designed to squelch creativity and how to workaround some of those structures.   And I'll be giving totally non-cynical advice about how to use the testing that is required in so many schools around the world to actually help learning, critical thinking, and creative thinking.   Thousands of teachers do it.   I want to pass that on to those taking my course.

And I want to use the course to rally lots of ideas about better ways of teaching and learning.   I'm donating my own compensation for my course (if they will let me) to HASTAC so we can continue to keep it a free and open community that anyone can join and contribute to without paying dues.  I have misgivings about MOOCs as a form so I am going to experiment with as many ways this particular MOOC can be useful as possible and cross-subsidizing a free, participatory, online learning community dedicated to free participatory learning seems like a splendid way to start!  But I also hope it is a way to organize energies around the world around sharing ideas for the "future of higher education."  That's my evangelical intention with all my public lectures and Coursera's big tent can be just right for an increasingly vocal majority, not minority, around the world who are interested in closing the gap between "formal education" and "peer , connected learning."   And maybe we can generate enough really great critique of this system that we can move to the next step of finding better ways.

In the meantime, here's rule one of turning standardized testing or in fact any teaching into a good learning system for retention.  I believe this three-step bit of cornballadvice comes from some old Borscht Belt comedian, I'm told it's from Henny Youngman although I've not confirmed it, but it is great advice for any speech or any teaching unit:  (1) Tell em what ya wanna tell em.   (2)  Tell em.  (3)  Tell em what ya told em. 

I'm going to use the multiple choice tests in my ten-minute Coursera class in this way, as a great way not just to find out what students have learned but as a way to summarize what the unit is about....to tell em what I told em.

Here's a tidbit from the segment "audition"  I'll tape next week onon multiple choice testing.  I'll pass on advice from my friend Jonathan Sterne, a professor in media studies and one of the brilliant theorists of sound I know.  He teaches huge lecture classes at McGill University and spends a lot of time thinking about effective pedagogies even within that highly restrictive form.   This year he blogged about his workaround for the end of grade multiple tests that he knows, from his extensive reading, are limited.   His learning solution to a rigid form is simple, brilliant, effective. If you subscribe to the Chronicle of Higher Education, you can find his column here:   http://chronicle.com/blogs/profhacker/multiple-choice-exam-theory/45275for   To summarize, he invitest students to bring in one page of notes that they sign and hand in with the test. They could be an art exhibit! But the real learning opportunity is the cognitive challenge of reducing an entire course to one page, and organizing it for the text, not the test itself.

I'm looking for other great ideas I can share with Coursera students.  I'll also share all the ways that, if you have to take them, you can do better by knowing the assumptions embedded in the form. And of course there will be a segment on better, formative test methods (but not everyone has the luxury of choice).

Sometimes you CAN use the master's tools to break out of the master's system. In fact, often you can . . .    That's my goal in this upcoming "History and Future of Higher Education" course.  I hope HASTAC network members will contribute their ideas too so we can spread the word.  

82

2 comments

First, and this has been a low-level obsession for years, is to convert the files - which most state testmakers post after their exams - that include now thousands of questions into a GAME. Flash card questions that are purely from the states themselves, with a logon for individuals or small groups, and prizes for speed, accuracy, or collaboration - from corporations promoting their products and responding to kids/parents/teachers, is a very easy idea for anybody with more technical savy than I. Most of those questions are in pdf files, and they can be converted, easily, into interactive flash pages. With local sponsors, a school, district, or network of districts could make test-prep what it really is: a game, with a prize.

My second suggestion, and a little more instructional, reflects my observation of kids in computer labs doing the high school MOOC equivalent for recovery courses. Typically, they'd jump to the tests rather than bother with the lugubrious video presentations, and answer what they could, and then choose one of two options: they could skim the speeches to get the answers or they could do a google on the question itself. More often than not, after a few skimming experiences of turgid garbage, they'd switch to googling the question. For some of the automated courses - Pearson, et.al. - the anwers had already been posted on Ask.com, so, if they were in a hurry or not particularly curious, they'd choose what Ask.com respondents had already suggested. Gradually, they discovered that the suggestions were not always right, and it paid off to look over other Google responses before choosing one that reflected a concensus.

That was actually fairly good self-instruction. My experience with MOOCs is less direct, but should have close analogs. When the issue is "the answer," the process of instruction has already been ratcheted down to a low minimum. Students have fairly good instincts on how to play the teacher (or system) for the best benefit, and most of the kids with whom I worked had already dismissed the courses for which they had to face a "recovery" computer. In many, many cases, they found this process less painful than 14 weeks of a boring teacher on a subject (which may or may not be "Common Core") for which they could see no direct utility. Over the two or three years I observed these kids, they'd create teams, organize themselves to scan lots of tests and questions, and beat a system that was already moribund.

Given the corruption of the testmakers, the standards advocates, the pseudo academics, and the paucity of project based inspiration, kids do find ways to get through a system fraught with roadblocks rather than inspiration. As Louis Agassiz said to his freshmen biology students for six weeks at every new term, having given them a fish direct from formaldehyde and keeping them from any incisions, "watch your fish." So, in like fashion, we ought to watch the kids. We've certainly failed to help them with the system now in place.

58

These are excellent suggestions.  I will be proud to pass them on with due credit---and, yes, yes, watch our kids.   We've certainly come up with a terrible system and need to help them thrive in any and every way we can.  Thanks for your constant contributions towards that end!

66