We asked retired study section chairs to tell us the advice they would give to new reviewers. The nuggets below come from many years of combined experience as reviewers and chairs.
Getting Started
Don’t be overwhelmed by the number of applications. Just start reading as soon as you get your application assignments. Give each application adequate time for a full and comprehensive review.
Get started right away to discover if you have a conflict with an application. It is cumbersome to reassign the application late in the game. You want to make sure every application gets due diligence.
Remember you are not alone: You can clarify any uncertainties about the review process with your scientific review officer (SRO) prior to writing your reviews.
Writing Your Reviews
Don’t just express your opinions: Be prepared to explain your opinion as objectively as possible.
Don’t mince words: If you feel there is an immutable and fatal flaw, say so in your review. Otherwise the investigator may needlessly move the deckchairs and resubmit the Titanic.
Don’t be too critical: Some people just come to the table and put their hat on to criticize. But an application doesn’t have to be totally perfect for you to give it a good score.
Paint a big and balanced picture: Sometimes it’s quite easy for inexperienced reviewers to focus on the weakness of things without providing a balanced review. You need to say what the strengths are and explain the big picture.
Applicants are just like you: They are excited about the science and often it is their life’s work. You really want to respect that.
Recognize that applicants can’t provide all the details in the shorter NIH format. Consider if the PI has sufficient command of the material to make it likely the project will succeed.
Don’t try to rewrite an application: You are a critic, not a mentor. Your comments should be generous when possible and polite at all times. They should be clear enough that an investigator has a sense of what needs to be done in order to craft a more competitive application if the current version is unfunded. However, your kindness should not extend to rewriting the application. Please don’t suggest better methodologies or designs, or salient references.
Score the application, not your discipline: Try not to root for work in your own field, and don’t inflate your score because you think there isn’t enough funding going into an area.
Keep your reviews in sync: Make sure that your comments and what you say are calibrated to the score that you’re giving.
Prepare the notes for your assigned applications well: So you can be conversational and efficient at your meeting. You don’t want to read your critiques.
Participating in the Meeting
Remember you’re not expected to know everything: Peer review is a group process with multiple individuals providing special expertise and different points of view.
Let Go and Have Fun: Be very clear about the strengths or the weaknesses and try to give a sense of what you think will be learned by things . . . And don’t be afraid to say there was a part of an application you didn’t really understand. [Alert your scientific review officer ahead of time if you feel unqualified to review an application so he/she can ensure it is reviewed thoroughly.] Don’t be afraid to say you felt most comfortable commenting on one aspect of a proposal or that you didn’t feel as comfortable evaluating another aspect. And don’t be afraid to change your mind—it’s not a turf war. It’s about being fair in evaluating the science. If we keep this focus, it’s both fun to do the review and scholarly.
Talk about what matters: Oral presentations at study section are not like anything most of us have done before. Study section time is valuable, so it is vital that the oral presentations focus on issues that help people arrive at a score. Sometimes the science is so good (or bad) that a reviewer wants to tell everyone all about it. But this brings down the energy level in the whole room and often diminishes our momentum. Just focus on the major strengths and weaknesses. Limit your description of the research and don’t try to enumerate every minor flaw.
Give key historical facts: If you are reviewing a resubmitted application, let us know if the investigators were responsive to the initial review group. If you are reviewing a renewal proposal for continued funding, please comment on the progress the investigators made.
Don’t rehash: If you agree with everything the previous reviewer said, say so and then say what you see as different. Too many people who have worked hard on their reviews want to give a 15-min or 20-min presentation that will say exactly the same thing as the person before. This causes frustration on the panel.
Focus on the applications: Always remember that the study section is about the applicants, and not about you. The first thing that I heard at my first major review was “The purpose of this meeting is to make decisions regarding the relative merit of the proposals, and not to show the committee how smart you are."
Focus on the big picture and not the little details. Is the application transformative? Or does the proposed research address an important need, even if it is not necessarily transformative?
Don’t be unsettled if someone disagrees with you: You were asked to be on the study section for a reason. You should feel comfortable sharing your expertise and knowledge. And it is OK to disagree.
Don’t be intimidated by a senior reviewer who comes down on an application you feel is terrific: You should be sure your opinion is stated, and it will be.
Be flexible: Listen and be open to changing your mind if new and compelling perspectives come to the table.
Participating in the Meeting
Meet new friends and colleagues: You will spend a lot of time with colleagues who are well-known in the field, and you have an opportunity to develop a community of science network through these relationships.
Enjoy the experience: You have the opportunity to see firsthand cutting-edge science and where a field might be going, you’re going to learn a lot about your field of science you wouldn’t otherwise learn, and you’re going to become a better grant writer as a result.
Learn More at NIH's Reviewer Orientation Site
