Today my students in "21st Century Literacies" will be holding a Q and A session with Professor Kathleen Fitzpatrick after her noon talk on "The Future of Authorship" at the Franklin Center. In preparation for this Q and A session, the two student peer leaders assigned several texts, led a provocative and engaged class discussion, and challenged students, as a writing assignment, to assess peer review, and then come up with some questions for Professor Fitzpatrick. Their class blog is so good, it raises so many questions, that I'm repeating it here, without their names in order to maintain their privacy. They are not accepting facile or glib answers. I know today is going to be a great talk followed by an engaged, exciting discussion. Join us!
Peer review its an English thing. Last semester I participated in a creative writing course, and I was reintroduced to the peer review. The previous semester in Writing 20, I also was subjected to the peer review. However, this is different, and yet analogous, to the peer review we discussed in class. In writing and English courses, peer reviews critique the grammar and style of writing rather than the content. Your peers arent analyzing the document to determine its accuracy but rather commenting on the writing stylistically. However, I can use this type of peer review as an analogy. The experience was always dreadful: to submit your work so your peers and professor could discuss (read: destroy) your work in front of you. But, in a sense, grammar is a science, and your peers arent experts. Yet you still (or should) value their feedback because they represent a smaller sample of the larger audience. With an open mind, I considered the comments the other students would make. Likewise, I think peer review, though allowing for non-experts to comment, permits useful feedback. I put non-experts in quotation marks (again) because this begs the question: what is an expert? Someone with a degree? In modern academia, a degree is everything but with websites like Wikipedia, people can be experts without the paper. These titles are debatable and so is the cut-off point, as we discussed in class. Thus, I think this topic can be effectively argued either way, but ultimately I think there is value in crowdsourcing.
But, the article brought up a point I had not considered: familiar work. The standards people hold our work to are rigid, tried and true. And, if people are working from a similar mindset established by culture and time period, how can we ever progress? If we listen to the masses or predetermined rules, nothing would advance. So, though we should consider the feedback of our peers, I dont think all feedback should be considered equal. Similarly, the academic world is also ruled with pedantry. And this opens up a new can of worms: when are they being helpful or when are they stifling our creativity? There is a fine line between the two, and it can be difficult to ascertain. I guess thats the asterisk or small notation beneath my previous paragraph.
Being not-so-tech savvy the term crowdsourcing was foreign to me so I looked up some other buzzwords. And I came across this:
On the surface, the two terms seem almost identical, but citizen science implies something that I had not considered previously. Because citizen science includes, well, citizens, these people have a vested interest in the project. They arent just editing or commenting on something, theyre apart of a project. The same principle could be applied to crowdsourcing, though. These people are now apart of something and that is a motivator in and of itself.
Question for Kathleen Fitzpatrick:
The website is described as between a blog and a journal. By journal, do you mean academic journal? And assuming you do (which I did), does that mean you think the credibility is somewhere between a blog and a journal, too?
The concept of peer review interests me, especially as an English major. If I reached the ranks of scholarship and were able to publish a journal article about an English Literature text, I feel that peer review would be something I would really want for my article. Analyzing a literature text can be done in so many different ways. In essence, English literature is an ongoing discussion, a never-ending review of what has been said. I guess what I love about reading texts is that there is never an answer to the question What does this mean? or What did the author intend?. Therefore, when writing in the field, peer review is something I have no issues with. Anybody, regardless of their level of scholarship, is qualified to make a judgement, analysis, and/or interpretation of a text. Whether or not it truly furthers the quest to discover more about the text does not really matter; rather, the field appreciates the contribution, I feel.
While writing this, Im now considering if peer review can exist or has value in art. For example, is peer review a good/beneficial thing if an artist wants to examine how valuable or improve-able a painting is or if an author wants to do the same with a poem or a novel? It seems that this kind of does exist, especially when an editor goes through a novel that is given by an author. Whether or not the editing process can be called peer review or if it can be analogous to the peer review process is something we can talk more about. Regardless though, to a certain extent, I feel like this process devaluates and/or decreases the purity of a work of art. Ive obviously never published a book, but I wonder how influential an editor is before a novel is published. How much does the novel change from when it was given to when it is published due to editors? If they do, in fact, have a huge influence in what is finally published, the question of authorship should be thought about. Does the novel, then, to some extent belong to the editor, even if they did not write the actual text? Ideas, suggestions, and changes, if taken into account, inevitably become a part of the novel. I think we could have a really interesting conversation about this in class.
Questions for Kathleen Fitzpatrick:
What do you think the future of the internet is in connection to scholarship?
How do you define collaboration?
At the risk of making these blog posts incredibly redundant, Im torn on the topic of peer review. On the one hand, should just anyone be able to critique work that someone spent a couple years of their life creating? Would it really be okay for Joe-Schmo to give you his corrections on your Biochemistry manuscript even though you have a PhD in Biochemistry and Joe dropped out of college? There is undoubtedly value in knowing what you are reading has been viewed by highly qualified members of academia who have given it their stamp of approval so to speak. I suppose the definition of qualification is where the confusion stems. Who do we consider to be qualified? What do we consider to be valuable qualifications?
This is not to say that I dont think the less qualified people can have very valuable contributions to pre-published works. Not only can their input be valuable but they can give a perspective that academic peers may not be able to give. As professor Davidson mentioned in class, often times we are unable to answer some of the most basic why-questions a four year old may present us with simply because we have grown to accept that this is just the way it is. We lose the concept of the process or the reasoning behind something because we grow accustomed to the results. In the same way, scholars often forget how to present the basics in a way that is easy to understand because they no longer need to do this for themselves. The public could help scholars present their work in the clearest, most accessible way.
On a slightly different note, I found this article on the topic of bias in the peer review system:
Interestingly, it states, reviewers were strongly biased against manuscripts which reported results contrary to their theoretical perspective. My first reaction to this was Close-minded jerks However, in reality, these results were probably not reached by experimenters surveying the reactions of a group of hard-headed people to extremely radical texts. A lot of reviewing is subconscious. It is not something being consciously controlled. In this sense, I agree with Kathleen Fitzpatrick on the idea that reviewing should be done by a much wider range of people. Who makes up this wider range is a less obvious.
Questions for Kathleen Fitzpatrick:
1) I think someone has already asked this but Im curious as to whether or not she feels scientific texts and texts in the humanities should have different methods of peer review.
2) Do you expect to see a lot of resistance to your efforts to change peer reviewing?
Our discussion last class brought up a very important issue that needs to be developed in order for crowdsourcing to become a reliable and credible tool for academiait needs a grammar. I believe we are at a moment of crude grammar. In language and in art it is grammar that directs and conducts us. With languages, it is not the word themselves that provide meaning, of course there is diction, but what is diction without syntax. Meaning, implication, and substance are achieved in the usage and the manipulation of its grammar. Grammar provides direction, misdirection; it orchestrates; it is the command; it is the question; it is the statement; it is the true tool of form. Im speaking rather abstractly but I think there are a lot of great parts to crowdsourcing and peer review, but I believe it needs a prudent sense of trajectory. Also, I am not insinuating a form of constraint on how it should operate; grammar does not constrain its language but imbues it with overflowing utilization, provides it with innumerable means of exercise.
Right now, based off the articles we read, the most effective means of peer review are ones done with a refined pool of reviewers. That doesnt mean it has to be a small pool, but I do think open crowdsourcing is not entirely productive. It has proved useful, but with such a vast amount of access it becomes less about the work and more about digital interaction. I honestly do believe the larger the pool the more the project begins to drift towards a focus on peer-to-peer interaction than on the project itself, the exchanges may be about and for the project but its the exchange itself that has now become the point of focus. This is fine but when reading over the Shakespeare Quarterly article there was more emphasis on how the exchange worked, the quickness of the exchange, and how the exchange was insightful. I do not care about this. It should be about what is best for the piece and the writer(s) of that piece. After so many eons of human history, it is apparent that, given the medium or platform to do so, human beings, being social creatures, have an adept ability to produce constructive concepts through exchange. Why are we so fascinated by it then?
Claudia brought up Zuckerbergs comment on the online experience being a social one. Now I may be getting a bit figurative here, but it brought up the question of, which I will direct to Kathleen Fitzpatrick, What is real human connection and relation? How has the digital world changed this? Has the change been positive or negative?
I play video games online and chat with friends online, but thats superficial to me. There is no depth to it. Having been reading a lot of D.H. Lawrence he is strongly inspired by what he calls the sensuousness of man. Is the online world capable of conserving this so called sensuousness?
How does text, audio and video compare to its immediate corporal counterpart?
Authorship is something that I feel is always going to come with a conflict. You may think that you have come up with such a great idea and may be the next big thing but, lets be honest in todays world and with the power of the internet someone has probably beat you to your idea. I saw Chloe mention it in her blog, but shes right The Social Network is a perfect example of this. What exactly does authorship or copyright law consist of? If you take someones idea but add some of your own twist to it does that mean its your idea? Is enhancing someones idea like using another authors work in a paper? Youre using cited work to make your own argument. If you give someone their credit for an idea, but add some of your own stuff to it why should they get a cut of your new idea?I feel like there is a fine line in authorship and it is constantly being tampered with.
As regards to a question for Mrs. Fitzpatrick. After reading the article posted by Jenny and Sarah. Googleing Peer Review, my question is clearly peer review is fundamental to academic and scientific research. What role if any have peer reviewers who were actually gatekeepers historically played in suppressing research?
Thinking back to our discussion in class this past Monday, Im still not completely able to find a strong standpoint on the idea of open peer review to a wider public versus the classic academic peer review process as it stands now. I do believe that with technology, internet, and the new information age that we are in now that changes should be made to accomadate the fast growing and ever expanding progression of knowledge. But where can the line be drawn, if any, for what is considered academic worthy work versus an uncredited source. Even using the word credited or uncredited merits an explantion as to who is able to set these standards and how can they be met. I think the way that the MediaCommons website allows for a larger pool of perspectives to comment, analyze, and review work (though not traditionally how a peer review is made in the academia sense) can actually lead to an even more credited piece of then if the traditional smaller group of people were only allowed to input their thoughts. It brings about the idea of collaboration and how through collaboration something can develop farther through the collective experiences of all of those involved. Why wouldnt we want to use the resources available to us? The resources of each individuals knowledge and own, unique perspective that can bring something different to the table. Without these varying viewpoints, how would anything new be developed or come about?
On a different note, I wanted to comment on the link about how Google works. I found it very interesting to think about how knowledge we search for on the internet is filtered. Actually, all knowledge we receive in some sense of the word is filtered, whether that filter is through the editors of a book, a professor who choses certain sources for a class rather than others, or even a certain environment someone inhabits which can also be a source a knowledge.
Lastly, I thought that I would share these two articles with everyone since they intrigued me a lot today (and actually distracted me during my BME 153 lecture this morning). They were sent to me coincidentally, but I think they can apply to this conversation weve been having. The second article is a great example of someone peer reviewing an article. I think it has many valid points and brings about a great perspective for looking at the first. Actually, even the first article is a review in of itself of some sort, on a probably controversial topic that Im sure everyone has heard of (Karen Owens power point). But regardless of the topic, it made me think about how information is passed on to people and can be portrayed in various ways in order to put it in a different light.
Question for Kathleen Fitzpatrick:
1) Do you think that there should be a different standard for how peer review is conducted of a humanities work versus a science or mathematics work? That one should be looked at in a different light or treated differently than the other because of the type of knowledge it is trying to present, or that all knowledge should be able to be scrutinized and reviewed in the same manners?
Our discussion brought to mind many memories from my FanFiction days, particularly concerning ownership, plagiarism, and peer review/feedback. I remember the incident in which another FanFiction author developed the same exact idea for a story as I did without copying me or even reading my story, introducing the role coincidence might play in such creative processes, or even highlighting the possible common inspiration we both had that led to us producing such similar pieces. I also remember another instance in which an author literally copy and pasted one of my older stories word for word and posted it as her own. I reported the abuse to the FanFiction site, and they promptly removed her post, but at the time I wondered if I truly had any right to my own story to begin with. I wasnt a published author, and the entire basis of FanFiction is to write stories using characters and plot events from books already existing. We were obliged to add a disclaimer to every one of our stories giving the original author credit with their material, and stating that we were simply borrowing their ideas for our own amusement. If that was the case, Im not sure I had any true right to my own story if everything about it was already created by someone else.
Another critical component of the site was the opportunity to leave comments, both anonymous and signed (though you could change your settings to only accept reviews from signed sources- I chose to accept both). I for one found the comments enormously helpful (for one thing, it boosted my ego as a budding writer to hear so many compliments on my writing abilities and how closely I could mimick the authors own voice), as they were direct feedback from the general public and the target audience. If I were a published author, I think I would value the feedback from such places (both anonymous and known) equally with the reactions of critics and book reviews, as the comments come straight from the readers their honest, straightforward opinions about the quality of the book rather than having all their feedback filter through the upper echelons of the critical world. While feedback from the experts helps promulgate a publication so it reaches more viewers, I still believe the millions of other voices deserve a chance to be heard. For this reason, I support initiatives like HASTACs willingness to accept feedback from all sources and the Shakespeare Quarterlys trial experiment with revolutionizing the world of scholarly peer review. I believe theres no true way to ever knowing where the most helpful comments or criticisms will come from, so its therefore important to be open to all venues (anonymous and signed, amateur and expert), given the facility of doing so in todays world of advanced communication.
While perusing the Internet for similar articles that broaden the ideas of authorship, copyright, and peer review, I came across this website detailing many of the overlapping and indiscernible boundaries of authorship in scientific initiatives. At the end of the first webpage theres the link: Answer Challenge Questions. This exercise provides a meaningful way to ask yourself what you believe the boundaries of authorship are and the ethics involved. After completing the quiz, you can see the results for other people whove taken the questionnaire and get an overall idea of the general publics viewpoints on the subjects of authorship and peer review.
Question for Dr. Fitzpatrick: The mantra for The New Everyday on the MediaCommons website is publish then filter. What do you feel are the particular advantages of publishing work and then editing or filtering the content as the second step?
The concepts of authorship and publishing are in flux. The first question we tend to want to ask ourselves is whether we are we progressing in a good direction. Is it a good thing for the people outside academia to critique scholarly articles? Should we rethink the current method of peer review? Should we rethink the word scholarly? Or how we think about expertise? There are clearly pros and cons to both the traditional and more open methods of reviewing academic papers. The concept of good is rarely black and white. My general feeling, however, is that we should spend less time considering whether this direction is good or bad, and more time considering ways to use these new methods to academias (or any fields) advantage. My belief is that the Internet is changing the world whether we like it or not, so why dont we make this change as positive as possible?
I am probably biased in this view by my current fascination with the recent movie, The Social Network, and the countless articles and interviews about Facebook and its history that I read subsequent to watching it (three times). Not only did I enjoy the movie, but also I was (and in part sadly still am) on a mission to figure out the real story of how this multi-billion dollar company came to be.
Everything I read and watched was fascinating, but my frustratingly obvious conclusion was this: its unlikely that Ill ever get the real story (even if I personally interviewed all involved parties). However, I did learn a lot in the process, and developed an understanding of why Facebook is so successful and why Mark Zuckerberg is such a genius. All people really care about is interacting with other people. Zuckerberg believes that the entire Internet is moving toward making every experience online a social one. He points to gaming as one area in which it is clear that the experience is becoming more interconnected. People dont want to play individual games on their individual Nintendos anymore. They want to play with their friends online.
Which brings me from my slight tangent back to my point. We have no say in whether academia will remain in its esoteric bubble or progress with the rest of the world. As Facebooks success makes clear, people want to connect with other people and the Internet is a way to make this collaborative process quick and easy. And as the Googling Peer Review article illuminates, large-scale peer-review can be extremely beneficial. Kathleen Fitzpatricks Media Commons is yet another example of people moving toward this interconnected and less formal setting to collaborate and interact. With the stipulation that we need a way to maintain the quality and accuracy of the work, this new form of collaboration (which is coming regardless of mine or anyone elses opinion) can only mean good things.
Questions for Kathleen Fitzpatrick:
Where do you see scholarly publishing in 10 years?
In your response to Mike OMalleys post, you consider the problem of how to interpret silence. What do you see as a resolution to this problem that may arise in an open forum for review?
I dont believe this link was among the ones listed in the Google Doc, but this is Kathleen Fitzpatricks response to Mike OMalleys blog post on peer review. Its definitely an interesting read and certainly worthwhile considering we will be listening to her on Wednesday:
As a biology major, Ive always thought about peer review in the context of the hard sciences and in the context of a scientist evaluating the research and results of a peers paper. Ive never actually thought about what the differences between peer review in the humanities and peer review in the sciences are. Although the methods of evaluating scholarly work in the two areas probably differ, they certainly are not mutually exclusive. I am likely to be generalizing quite a bit here, but I think the main difference lies in the fact that scientific research papers usually have definitive results acquired from specific research methods, while humanities papers rely mostly on contextual analysis and take on more of a persuasive tone.
I think crowdsourcing can definitely be a more effective way of peer review. Actually, I should clarify exactly what I mean by effective. I think the only guaranteed thing that makes peer review via crowdsourcing more effective is that it would take less time. That doesnt even necessarily mean that the whole process will be more efficient. With that said, however, the example of the proposed mathematical proof of P vs NP is a good example of the efficacy of crowdsourcing peer review. When you have many experts in the field, in this case, mathematics, put together, it certainly increases the efficiency of the process because you have all these great minds thinking together. Like the article aid, this crowdsourcing peer review worked well because they disproved the proof in the scholarly equivalent of warp speed.
Then again, however, mathematical proofs fall more or less into the hard sciences where you can essentially say yes or no to a proposal based on research and other evidence. In my opinion, with the Internet at the disposal of peer reviewers, crowdsourcing as a method of effective peer review should definitely be considered. With only experts in the field as members of the crowd, there would not be any inane comments made such as those made in the example of crowdsourcing art on Larry Cebulas blog Northwest History. Keeping the process double-blind would certainly benefit the process. In addition, the peer reviewers in the crowd should not know the identity of the other experts, as this may influence their comments and evaluation.
Having read Kathleen Fitzpatricks blog post response, I am really intrigued by what she honestly thinks about peer review. She doesnt really take a firm stance on either side of the argument in her post and delivers arguments on both sides. She says shes no apologist for our current peer review systems but she also doesnt want a situation in which a community of the like-minded inadvertently excludes from its consideration those who dont fall within their sphere of reference. So these would be my questions:
- What are some of the biggest problems with scholarly peer review as it is now and how do you foresee future changes, such as crowdsourcing, affecting the process?
- What is the process of peer review or the process of being peer reviewed like for you?
After reading through the articles Jenny and Sarah posted for this week, I cant help but notice that (for the most part) the experimentations with digital peer review and crowdsourcing have been wildly successful. Based on my own experiences, Im not surprised in the least.
Last year, when I applied to be a writing tutor for the Writing Studio, one of the questions on the application asked what type of feedback I value most highly as a writer. This is what I wrote:
To me, the best feedback forces me to delve even deeper into my argument and ideas. I have found that this generally comes in the form of a question whether they are clarifying questions or simply probing questions. Often, by answering my readers questions, I automatically elucidate and broaden my own thoughts. Ideas and phrases that were previously incomplete or confusing become clear and thorough.
That response very nicely ties into our discussion of peer review. In the world of traditional academic peer review, reviewers are experts in their field. In one sense, this makes them the most qualified to check for factual errors and ultimately decide whether or not a piece belongs in an academic journal. However, I think it also makes them unlikely to really question a piece so much as simply accept or reject it. In my opinion, sites like MediaCommonswhere not all reviewers are expertsare much more likely to elicit meaningful questions from readers. Just as we talked about in class with tutors really having to struggle to explain their ideas to a tutee, this process should eventually help authors tighten up their arguments and create an even better final products. Whether these products will be accepted as legitimate by traditional academia remains to be seen.
I want to transition away from peer review and say a little bit about authorship. I know we talked about this briefly in class, but The Social Network brings up some really fascinating points about authorship. For those of you who have yet to see the film (you should!), two students at Harvard sue Facebook founder Mark Zuckerberg for stealing their idea. Zuckerberg defends himself by saying that, though the idea of Facebook is the same, he used none of the same code. In a digital age when ideas are so much more than just words on a page, the concept of authorship becomes even more difficult to define. Zuckerbergs code was entirely his ownand yet he still paid millions of dollars to the twins to settle the lawsuit. Who is the author of a websitethe person who wrote the code or the person who came up with the concept? Its definitely something to think about
It should be very interesting to listen to Kathleen Fitzpatrick speak about some of these topics tomorrow, as (based on my research) she seems to be one of the pioneers in this peer review revolution. The one big question I have for Ms. Fitzpatrick is: What do you envision as the future of academic peer review, and do you see sites like MediaCommons having any tangible effect on the current traditional process?