Creating Good Peer Review

My computer is acting funky, at the moment, so I am backing everything up and taking it to the Apple Store this afternoon for a check up. But, since I can’t do any “real” work at the moment while I wait for the back-up to finish, I thought I would do a post on how I teach peer review.

I learned the value of peer review at UConn, where a core feature of our composition courses is the Small Group Tutorial, or SGT, where the instructor would meet with a group of two to four students for an hour, and lead a peer review session. I didn’t like required peer review as an undergraduate, because my own experience was that I didn’t get good feedback; I did much better when I sought out my own readers and exchanged papers with people in different classes, or even at different universities. But SGTs helped me reorient my thinking; I started to think about including peer review not just because it helped the writer, but because it was about training students to do peer review. Rather than expecting students to already know how to give feedback, I was teaching and modeling it for them.

However, the teaching load at most schools really isn’t conducive to hour-long sessions with small groups of students, so at Georgia Tech I’ve been experimenting with how to teach peer review and make it more effective for students. I’m focusing on teaching what peer review is, why we do it, and what it (should) be good for. What I did this semester worked surprisingly well, and a few colleagues here and at other schools asked me for my materials, so I thought I would share what I do. But first, two notes:

1) As I said in my last post, I regularly use Joseph Harris’ book Rewriting in my classes. A lot of the language of this prompt is drawn from his book. Students are already familiar with the terms when we start peer review.

2) For the first round of peer review, I didn’t have students meet together or give comments to each other orally.* Instead, I have them do double-blind peer review, and later I use a discussion of how that worked to set up a discussion of academic research, and how academics use peer review. I use Piazza, which allows me to set up numbered discussion groups as well as anonymous posting. I assign each student a number, and each number to a group, and they post all their papers there. There were at least three students in each group, so that even if one member of a group didn’t give comments, everyone would get something. I also added extra office hours so that if anyone wanted help figuring out what to do with the comments they received, I would be available.

My students get two sets of instructions, one that is built into our class assignment guide, and then a reiteration/extension of those basics, usually in the form of a class lecture/discussion. However, we had a lot of snow days this year, so my students got the second set in an e-mail, which makes it a lot easier for me to share them with you here.

Write an anonymous letter to the authors of the papers you reviewed, including but not limited to the following:

    1. Come to Terms: Explain what you understood as the project of the paper. Start by identifying the thesis, but also give a holistic explanation of what you see the author trying to do.
    2. Forwarding: What works in the draft? How can the author build on and do more of that?
    3. Countering: Where are there places that the logical argument and/or use of evidence could be improved? How could the author acknowledge other points of view or other possibilities? Where do you think, “That’s not what I thought when I read that,” or “Maybe, but what about X?”

Your peers will grade you on the quality and usefulness of your responses, so be sure to put in sufficient effort. Some tips to make your comments more effective:

    • Don’t ever say “this is a good paper.” Instead, identify specific elements that work well.
    • Specifics should be paired with suggestions for improvement. For example: “I really didn’t understand how your quotation of Satan’s appeal to Eve’s beauty in the second paragraph had anything to do with your point that Eve’s behavior was unselfish. Could you find a more appropriate quotation, or explain what you were thinking a bit more?”
    • Quote the paper: “Your thesis in the introduction is that ‘Milton’s God is cold and unfeeling in order to make Satan seem more appealing.’ However, most of your paper actually seems to be about Satan’s perception of God, which isn’t the same thing. Maybe you should rewrite the thesis to better reflect the contents of your paper? I think Satan’s perceptions are more interesting than your original thesis.”

For the in-class or e-mail instructions, I discuss broader issues of why and how we approach peer review. I cover at least the following points.

  1. The anonymity of your drafts and peer review comments is designed to give you freedom in responding: freedom to know that whoever comments on your paper won’t know who you are, freedom to express your confusion about something someone has written, freedom to be critical, etc. You won’t always have this anonymity, but it can be useful when you are first learning how to give feedback.
  2. In addition to thinking in terms of Forwarding and Countering, make sure to think about the rubric that I’ll be using to grade the papers, which is included in the assignment guide. Forwarding and Countering are ways of helping to improve ideas, but you don’t want to forget about the rhetorical elements we’ve been discussing in class.
  3. Your job is not to be “nice.” Your job, as an ethical reader of someone else’s work, is to help them revise their paper into the best it can possibly be. You should be professional, of course, but it is most important that you be honest about what doesn’t work in the paper. One of my favorite fantasy novels, The Curse of Chalion, includes this line, from a teacher (Cazaril) to his students:

“If you desire a man to tell you comfortable lies about your prowess, and so fetter any hope of true excellence, I’m sure you may find one anywhere. Not all prisons are made of iron bars. Some are made of feather beds.”

The point Cazaril makes is that telling someone their not-so-good work is good is easy, but doing so does them a disservice, and may actually hurt them. It binds them, and keeps them from improving. It prevents them from being truly excellent. Even if the paper you are commenting on is better than yours, it doesn’t mean you can’t offer them good feedback–you might still see things in their paper that they can’t. Don’t stop with identifying what is wrong, however; make suggestions for improvement. And even when someone does something that works, try and help them make it even better.

Then, finally, I have students grade the peer review letters they received.

Rank each peer review letter: give a “5” for a letter that was very detailed & helpful; a “4” for one that was somewhat detailed & useful; a “3” for a letter that was useful but not detailed; a “2” for one that was detailed but not useful; a “1” for one that was neither; and “0” for one not submitted.

I was very pleasantly surprised by the quality of comments most of my students produced. Most students did light editing/marginal notes using Word’s reviewing features, and then wrote several paragraphs focusing on the big picture issues. A few students actually full-page letters with comments–in other words, for these 6-8 page papers, they were giving more comments than academic peer reviewers usually do for 20-page articles. And, to be honest, taking into account that these were freshmen papers and freshmen reviewers, the quality of the comments was sometimes higher than the average journal peer review response.

When I asked students about what they liked about this system, a lot of them said they appreciated the anonymity; they felt freer to actually give criticism instead of just saying “good job”. Most also liked that because the letter was the primary form of commenting, it meant the focus was less on editing and more on true revision. But the comment that I found extremely telling came from a particularly bright but somewhat disengaged student, who admitted that in the past, he had ignored peer review comments unless they came from someone he thought was smarter than he was. The anonymous process deprived him of his reliance on perceived authority/intelligence, and forced him to really pay attention and decide based on the merits of the comment. I saw enough nods around the table at that one to think that might be a bigger issue with traditional peer review than I had previously thought, at least with students at Georgia Tech, who are often used to being the smartest among their high school peers.

This is still a work in progress for me. Not all the students bought in; there were a handful who didn’t take the advice of their peers, or who didn’t give good advice. This process also doesn’t really get at the dialectic or collaborative elements that I truly valued when I exchanged work with peers of my own choosing (with long sessions on IM as we figured out how to rewrite paragraphs and reorganize documents). I want my students to get that experience, too.

So: What do you do for peer review that works well? How do you teach good peer review skills? What do you emphasize?


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s