Blog Post

How to Crowdsource Grading: A Report Card

So last July I wrote that "How To Crowdsource Grading" post about how some of my very best students in "This Is Your Brain on the Internet" praised the course but also got together and, with care, raised the delicate matter of my o-so-conventional ways of testing and assessing and grading:  midterm, final, final paper, A, B, C, D.  You know the drill. 


Being a prof interested in exploration of pedagogy at every turn, I came up with a system of old-fashioned contract grading (we really did create a contract with co-signatures and all) and peer review to determine if all of the weekly blogging assignments had been created satisfactorily and, if not, to work with students to make sure a satisfactory one is posted.   A few weeks ago, I wrote one update, "Crowdsourcing Grades:  Or, How Prof D Got an A":


And here's another:  I was feeling a lot  of anxiety in the class, exactly what I hoped from crowdsourced grading where the outcome is known as long as you keep the contract in a satisfactory fashion.   I had no anxiety.  I've never seen such amazing attendance, there is lots of participation, and the blogs have a depth, engagement, insight, and intelligence that one rarely sees.   I've long since concluded that blogs are  better than research papers at getting to the heart of the matter. . . but that's a different issue that we're having a mini-forum on over here if you are interested:


So my anxious crowdsourced class?  I decided to hand out a midsemester evaluation to see how I could do a better job.  And found there were so many anxieties.  Without the familiar markers of midterms and grades, instead of feeling sanguine about the coming A, the students were sure I was going to pull a fast one.  Not that they said that--they are much too generous a group as human beings to say that.   But they were concerned, nervous, anxious.   So what Alex Greenberg, the able TA in the course, and I did was pulled together a synopsis of where everyone stood so far on their contracts, what needed to be done, and offered guidance on how to do what remained.   I swear the tension level dropped immediately.


So to those naysayers out there wondering about my crowdsourced grading, the issue (at least for these remarkable students) isn't whether they will flake out and do nothing.  It's that, given an entirely new way of thinking about one's responsibilities in a classroom, with a prof with whom one has no history, is there really going to be that coveted A at the end of the term or is this going to be a disaster?   There is no past experience, no test case, no track record, and no evidence.   How do you argue about an unfair grade when you have nothing to present on your behalf?   That's a lot of trust to have in a prof you've never had before.


On the other hand, there is that matter of the written contract.  That's what we did by letting people know, in writing, where they stood on each part of the contract.  We recalled that they did have some real, tangible evidence of progress.   Besides, they are exceeding any of my expectations--about attendance, participation, about the length and quality of their blogs.  Today we move into the multimedia collaborative projects, the TYBI equivalent of a final paper, and I'll report back on those but, to date, this class is A all the way, whether by crowdsourced standards or conventional ones, or any other metric I know.


But what about Prof D?  The one interesting thing about contract grading is it's the students who sign the contract, the prof countersigns, but it is of course about how they do and how they will be graded.  There's really no contract in the other direction, except the student evaluations (which is more like Pass/Fail rather than a grading system).    That's interesting.  And a subject I'll take up next time.  If students wrote a contract for what they expect from a prof, what would that contract look like?   And how would we measure up?   Now there is a challenge!


Here's the original blog that garnered all the notice:  "How To Crowdsource Grading":


And you can use the tag ISIS 120 to find other blogs about "This Is Your Brain on the Internet"




No comments