Top 10 Peer Review Q&As for NIH Applicants Final Transcript 9/27/2018 Running Time 14 minutes and 31 seconds AUDIO (Music up) NARRATOR: You need to know a lot to successfully navigate NIH peer review and get a grant. So we’ve gathered 10 experts from the NIH Center for Scientific Review to answer the top 10 questions about NIH peer review – plus a bonus question at the end. Let’s start with Number 10: How do I withdraw my application? INGRID LI: The signing official at your institution can now request withdrawal of an application directly through eRA Commons. See NIH Guide Notice NOT-OD-16-143 for instructions. Alternatively, a letter with ink signature from your signing official can be sent to csrdrr@mail.nih.gov and we will process your request manually. NARRATOR: Number 9: What do you mean by “scientific premise or rigor”? I’m confused about the terms and how they are different from “significance.” Can you tell me what reviewers are supposed to be looking for? TATIANA COHEN: Reviewers are currently asked under the Significance criterion for research grant applications whether there is a strong scientific premise for the project. In evaluating the scientific premise of a proposed project, the reviewer considers the rigor of the prior research that serves as the key support for the proposed project. The NIH is moving away from the term “scientific premise” for applications that come in for due dates of January 25, 2019, and beyond. In evaluating these applications, reviewers still will be asked whether the prior research that serves as the key support for the proposed project is rigorous. Scientific premise or rigor is just part of the consideration of significance, which includes consideration of the importance of the problem, critical barriers to progress, how the proposed project will improve scientific knowledge, and how the field will change should the aims be achieved. NARRATOR: Number 8: What is the difference between “significance” and “overall impact”? I’ve read the definitions, but the two still seem rather similar. Can you provide some additional guidance? MARY CUSTER: Significance is a stand-alone assessment of the project’s goals in the context of the relevant field, and to a large extent assumes that the investigator(s), approach and environment are adequate to allow for successful completion of the aims of the project even if later discussion of each of these review criteria will identify problems. When reviewers assess the Overall Impact of an application they are expected to take into account the scored review criteria (e.g., significance, investigator(s), innovation, approach and environment) and the additional review criteria to judge the potential of the project to exert a sustained, powerful influence on the field. For more information, visit the Overall Impact versus Significance document at www.csr.nih.gov/impactandsig NARRATOR: Number 7: I'm a reviewer, can I submit my application late? YUJING LIU: Most funding opportunity announcements (FOAs) have a two-week late window of consideration during which time an application can be submitted late. However, the terms are very specific, and don’t apply to some Requests for Applications (RFAs). Examples of reasons why late applications might be accepted include Review Service, Illness, Natural Disasters etc. However, no advance permissions can be given for late applications. You should list your reasons in the cover letter with your application and the decision will be made on a case-by-case basis. You should read the NIH late policy in the NIH Guide Notice NOT-OD-15-039 for an explanation of how the policy may apply to you. In addition, some reviewers have continuous submission eligibility because they are appointed to an NIH review group such as a study section or institute advisory council, or have “recent substantial service.” Under continuous submission, these reviewers may submit R01, R21, and R34 applications with standard due dates at any time during the council round. Read the NIH continuous submission policy in both NOT-OD-17-042 and NOT-OD-18-178 if this policy applies to you. NARRATOR: Number 6: Must I wait for my summary statement before submitting my idea again? SHIV PRASAD Once your application has been reviewed, you must wait for the summary statement to be issued before you resubmit that application or submit any other application with substantial scientific overlap. NARRATOR: Number 5: When will I receive word on my application? JACINTA BRONTE-TINKEW: Notification that CSR has assigned your application to a scientific review group and Institute should appear in your eRA Commons account within 2 weeks of the submission deadline. If this notification does not appear in this timeframe, please contact CSR’s Division of Receipt and Referral at CSRDRR@mail.nih.gov or 301 435-0715 After the review, your scores should appear in your Commons account within 3 business days, and your summary statement within 30 days. New investigators who submitted R01 applications should be able to access their summary statements within 10 days after the review meeting. It is important to note that funding decisions are not made until after the relevant Institute or Center council makes its recommendations. Phone number and email needs to be added to original on CSR Web. NARRATOR: Number 4: How do you guard against a single reviewer having undue influence at the review? JAMES MACK: Before the review meeting, the Scientific Review Officer or SRO looks for instances where one reviewer’s scores or critiques are out of sync with the others. The SRO will notify the chair and assigned reviewers when this happens. If the differences are not resolved before the meeting, the chair will make sure the study section discusses them to see if they are well founded or not. Also, if a review discussion is one-sided, chairs are trained to ask questions and encourage other reviewers to join the discussion. It is important to note that reviewers take their jobs seriously, and they routinely question each other when they feel a review is not balanced. In addition, all the reviewers in the room weigh what is said at the meeting and independently score the application. Finally, the SRO monitors the review meeting to ensure it is fair and unbiased. If it appears a reviewer is not meeting this standard, the SRO can pause the meeting to talk to the reviewer, conduct a re-review of the application at a later time, or take other appropriate actions. NARRATOR Number 3: I don’t like the review group you put my application into. What can I do? BRUCE REED: Contact the Chief of the integrated review group for the assigned study section. He or she will be able to explain to you why the assigned study section is the best fit for your application. CSR listens to PIs and considers their preferences in making application assignments. When you submit your application, you may (but need not) use the Assignment Request form to suggest up to three study sections. Helpful resources for finding a CSR study section are the study section guidelines, and the Assignment Request Tool. You can find both on our home page: www.csr.nih.gov. You also can use the Assignment Request form in your application to tell us the types of expertise needed to appropriately review your grant application. You should not request specific individuals. In addition, you can use this form to identify individuals who may be in conflict with your application. NIH will evaluate the situation according to our conflict of interest standards. See the NIH conflict-of-interest page: https://grants.nih.gov/grants/policy/coi. Please note that being competitors is NOT a compelling reason. If you suggest one or more study sections and we can find a good fit among them, we will assign the application there. If our referral professionals judge the fit to be poor, they work to find an appropriate alternative. Our paramount concern is having appropriate expertise on the panel where the application is reviewed. If you don’t understand, ask. In the great majority of cases, PI concerns about application assignments are resolved through communication. NARRATOR: Number 2: I addressed the concerns from the prior critiques and my score got worse. Why? NONI BYRNES: Each time your application is submitted, it competes within a new set of applications, it may be reviewed by new reviewers, or one or more of the original reviewers may see new concerns. So there is no guarantee your score will improve. The best course of action is to discuss your review concerns with your Program Officer before you resubmit and then be as responsive as possible to them. Including additional data, if available, could be helpful. In any event, if issues with significance and overall impact remain, addressing methodological concerns will not very likely improve the score. NARRATOR: Number 1: What are the biggest problems reviewers find in applications? LARRY BOERBOOM: Here is a list of the most frequent problems reviewers cite when they critique grant applications: * Lack of new or original ideas * Absence of an acceptable scientific rationale * Lack of experience in the essential methodology * Questionable reasoning in experimental approach * Uncritical approach * Diffuse, superficial, or unfocused research plan * Lack of sufficient experimental detail * Lack of knowledge of published relevant work * Unrealistically large amount of work proposed * Uncertainty concerning future directions For more insight, check out the reviewer tips for applicants in our Insider Guide to NIH Peer Review at http://www.csr.nih.gov/applicantresources/insider NARRATOR: Chances are you will have other questions. We’ve posted over 100 frequently asked questions about NIH peer review for you on the NIH Center for Scientific Review Web site: www.csr.nih.gov/faq. You’ll also find a wealth of information on the NIH Office of Extramural Research Web page: www.grants.nih.gov. If you have general questions about NIH grants and review, you can also contact GrantsInfo at GrantsInfo@nih.gov or phone 301 435-0714. But if you have more specific questions, here is some helpful guidance: * Before you submit your application, contact a Program Officer at an NIH Institute or Center. Or contact a Scientific Review Officer * After you submit, contact your assigned Scientific Review Officer * After your review, contact your assigned Program Officer Here’s the Bonus Question: How can I find a Program Officer or Scientific Review Officer to answer questions before submitting my application? MARY CUSTER: Before you submit your application, Program Officers or POs can identify the right type of grant program and/or funding opportunity for you and your research? and verify that your idea fits within the mission and priorities of an NIH Institute or Center. POs also can refer you to appropriate scientific review officers or study sections. To find a PO or an NIH Institute or Center that might fund your research, go to the Matchmaker tool in NIH’s RePORTER database at https://projectreporter.nih.gov and click on the find-program-officials tab. If you wish to find a scientific reviewer officer or study section at CSR, you can search study section descriptions or use the Assisted Referral Tool on CSR’s home page: www.csr.nih.gov After you submit your application, your assigned program officer and scientific review officer will be listed in your eRA Commons account. For more guidance, visit the Contact NIH Staff page at www.csr.nih.gov/NIHcontacts NARRATOR: We hope you found this information helpful and wish you well as you advance your application.