Testing, Testing! A Pointed Response to the NY Times

Since I am a passionate critic of the current state of testing in the United States, where No Child Left Behind bases  its judgment of students, teachers, and school districts on the so-called End of Grade (EOG) form of item-response (multiple choice) testing, some people think I am against testing.   Wrong!   I agree with the findings of recent researchers that testing is, in fact, a great way to learn.   Test early, test often, is my motto. 


But (yes, there is a "but"):  don't test simply to assess--test kids in order to help them to strive, to challenge themselves, and therefore to learn.   Let students see how they do sooner, not later.  Don't make it about cramming for a test.  Integrate testing challenges into the learning as a process.  Let them learn from the kinds of tests they excel at and the ones they don't.  Let them help to design the test.  Let them test one another.   And let them help one another--including by other challenges, other tests--to do better the next time.  That is the central component of real-time, in-process game-mechanic based testing.  And it works.  Always has, always will.


Here's the url for the New York Times piece by Pam Belluck, "To Really Learn, Quit Studying and Take a Test":  http://www.nytimes.com/2011/01/21/science/21memory.html?emc=eta1   


My advocacy of "crowdsourced grading" isn't because I don't want students to achieve.  It is quite the opposite.  All of the best research, whether on tennis or foot races or chess or math or English grammar or foreign language learning or critical thinking or World of Warcraft suggests you do best when you have to mount a challenge.   Challenging others challenges you--including in situations where you are working together towards a goal, towards a challenge.  "Let's put on a show!" was the motto of old musicals and it always works.   You set the goal, and you work together, and you achieve.   That is true of all forms of testing.


As we are learning, the forms of testing we have inherited are new in human history, not quite a hundred years for the form of multiple choice, item response testing that now is considered the "standard" for how we measure.   A lot of social science has gone into convincing parents who see their kids achieve remarkable things in the first five years of life, without a single multiple choice test to measure progress, that, once a kid hits school, learning needs to happen at scheduled intervals and in prescriptive fashion, reduced to a small circle colored in where the choices are narrowed to a very few options, some of them silly or nonsensical or trivial and only one of them "right."   Really?  Nothing about learning from birth to the advent of formal schooling suggests that is how we learn best.  And very little after formal education is over suggests that is what inspires us in our daily lives or in the workplace.


School has created artificial goals that assess us as individuals and groups.   Testing early and often, in real time, to understand what we have learned and what we need to learn,  is almost the opposite of the standardized end-of-grade form of testing for which one needs to cram or even to attend "cram school."   What you learn then is how to take tests.  What you learn from repeated, real-time, in process testing is how to succeed.


I vote for that, for the testing to succeed, every time.   We need to unravel the mystique of our current standardization metrics in order to understand more about the actual goals for testing.   Is the testing to ensure that each school meets a minimum national standard and that, even though schooling has historically been delegated locally, there are some national expectations about curriculum?   That is a worthy goal.  (Remember, Texas school boards this year wanted to demote Thomas Jefferson as a Founding Father because he was a deist.) On the other hand, the goal of a national bar for what kids should learn is very different than the pedagogical goal of inspiring learning and encouraging students to aspire to learn more, including in areas where they may have little interest or aptitude.  In those situations, the single End of Grade test is even more likely to lead to cramming, to studying for the test.  And given the punitive stance towards school districts and teachers whose students do not earn passing grades on these tests, the institution of the EOG is likely to encourage "studying for the test," the most deplorably circular component of our current ideas of testing.   That is, if you study for content, to help you learn, you reinforce what you know on build on it.   With the End of Grade test, you teach and study to the form of the questions and the anticipated type and scope of questions such that the test becomes the end in itself, rather than a means to greater learning.


We need to learn more about how kids love to learn, feed their own learning, test themselves and one another.   As usual, kids have a lot to teach us if we only stop thinking we have to make them all conform to a way of learning that is arbitrary and has little to do with how they learn on their owns and what they need to know to be able to learn for the rest of their lives. 

Evan Donahue

One thing jumped out at me:

One thing jumped out at me: “When you’re retrieving something out of a computer’s memory, you don’t change anything — it’s simple playback... when we use our memories by retrieving things, we change our access” to that information, Dr. Bjork said. “What we recall becomes more recallable in the future. In a sense you are practicing what you are going to need to do later.”” There seems to be a tremendous number of loose brain-as-computer or brain-as-not-computer or brain-computer comparisons running around these days. Apart from the fact that I think the whole principle is acritical in the extreme, I do wonder what the comparison says about the researcher's paradigm.

The statement about computers is wholly contingent on what the computer is doing. In point of fact modern computers do cache retrieved files in main memory, which seems to be what Bjork is saying is unique to human minds, but that's a technical detail I'll let go of. What I particularly wanted to observe is that if you only look at a computer (and mind as computer or not computer) as an electronic file cabinet, you privelege a certain interpretation of what the "information" buried in the computer consists of. What a file system is is static. Unaccessed, it remains unchanged. However, computers are far more than information retrieval systems. Computers do not simply access their own filesystems; they do so in the process of computing sensory input, calculating arrangements within main memory, optimizing their own disk geometry, rewriting parts of their own workings with downloads and upgrades. Access of a file system is ALWAYS subservient to another task. To understand a computer in relationship to its own access of its filesystem is to say that the information buried on disk rather than the input, networking, processing, rendering, and installing tasks is the important thing about a computer.

This model of the computer, when held against the human mind for comparison, leaves a human mind that is only responsible for the recall of information rather than the perceiving, relating, thinking, understanding and learning tasks as a metric on its performance. What is the recall of unfamiliar scientific facts a measure of? What task is the mind performing that such recall is the critical measure? The article seemed to me to be about that paradigm--retaining "material"--which made "material" or "files on a file system" the final metric of learning. The article pits something like "concept mapping" against whatever else and compares them in terms of this "retrieval-as-learning" paradigm. What I would suggest is that the mapping of unfamiliar scientific facts after brief exposure doesn't seem to me like something that can strike much at the heart of anything one might recognize as a "concept," and concepts are few and far between in current education as far as I can see. How about a study where concept maps, flashcards, whatever else are stacked up against an organized synthesis of intuitive concepts over the course of a semester? Are all types of material preparable in the same way? I don't think too many people would argue that a deep, fundamental systemic understanding that obviates the need for any kind of fact retrieval would be the ideal scenario and that fact retrieval is more of a more easily measurable and teachable relationship to learning, but it sometimes seems like, driven by practical necessity, we don't even try to talk about the former, which seems like a mistake out of the gate.



Cathy Davidson

Hal Remains the Metaphoric Computer

As usual, Evan, you and I are on exactly the same page.   My blog focused on the fallacy of one-time-only (EOG-style) testing as opposed to in-process, real-time challenges.  The former do not promote learning but promote an ends-oriented studying to do well on the test.  The former, if coupled with the right pedagogy, is very useful.    But, yes, I was struck by how outmoded was the metaphoric computer the researchers were summoning up.   Haven't they ever heard of Google?   Of course contemporary computers store previous results and learn from them.   That's what "search" is!   In fact, computers do that more reliably than humans since most of what we know about memory in humans is that it tends (and here's another contradiction in this article about testing as presented here) to be recombinant, in the sense that, we don't remember exactly in the same way every time but tend to pull out the parts of the memory that are useful for the moment, often reshaping what we think we remember to the particular situation (which can be anything from a job requirement to a stomach ache to a cloudy day:  "situation" is multimodal).   As you suggest, the paradigms here are too simplistic for human learning---or for machine learning!

Evan Donahue

of course that said, if what

of course that said, if what you need is factual recall -- and the situation may warrant it -- then by all means do what works for human recall. just don't politicize it as learning.