How to conduct a peer review.

In my experience, poor quality reviews have some common characteristics.  These are, in no particular order:

• An overall lack of effort to really read the manuscript and understand what it is the author is talking about.  Too often reviewers tend to try and crowbar the subject of a paper into their own paradigm instead of evaluating the paper on its own merit.
  
• Copy editing.  Generating a list of 17 insights, 15 of which are suggestions for restructuring the language of a sentence or paragraph is not useful to the author.
• Making a strong statement of fact without supporting evidence; or essentially offering the criticism "You're wrong, I'm right."
• Making contradictory criticisms; for example:  "This topic requires a broader and more specific discussion." 
• Personal nastiness.

Undoubtedly reviews are low priority endeavors for many research faculty because reviewing does not count much towards tenure.  The temptations to soothe oneself psychologically by paying negative reviews forward, or attempting to boost one's ego by demeaning the work of someone else are undoubtedly strong ones.  Then of course, there are the first-year grad students who think they've learned everything the need to know already - or even knew it before they got to grad school - and have to demonstrate it; and unfortunately end up like the blonde guy in Good Will Hunting:



But what can we be done to conduct useful peer reviews; even of submissions we do not particularly care for?  Here is what I've learned:

• Take the approach to reviewing that you want the paper to get published; that the review process is, to a degree, a collaboration between you (the reviewer) and the author to work to make the paper better.  Avoid the pitfall of starting out with the view that the paper is a failure and has to prove it is not.
• If you do not have expertise on a particular area then state that upfront and either make no comment or limit your comments to non-expert issues.  For example, if you do not know the details of a particular analytical method do not criticize it unless you are willing to learn it or deconstruct it and analyze the results.
• Do not offer revision suggestions that are, in essence, ways to re-frame the paper to one that you would write on the topic that you would write about.  The authors would have written that paper originally if they wanted to.
  
• Look for ways to offer suggestions that strengthen the paper.  For example, if you're aware of literature that speaks to the particular submission and may have some salience offer specific citations for the author to consult and reasons for why they might be useful.
• If the paper is in an area for which you are only tangentially aware, take the time to learn some about the area so you can make a more informed assessment of the manuscript's contribution.
• Finally, but most importantly:  write the kind of review you would like to receive yourself.  By this I do not mean in terms of accept or reject; after all it's self-evident we all want to have our work accepted.  What I mean here,  give your review in the style you would like to have, and with the depth of insight and content you would like to have.  Really think about what whether or not a comment such as "The conclusions are all common sense," really is the kind of feedback that would be useful if someone gave it to you.

That's my $.02 on reviewing.