Strategies to Strengthen Fairness and Impartiality in Peer Review

Challenge #2: Strategies to Strengthen Fairness and Impartiality in Peer Review

SUMMARY OF CHALLENGE: The National Institutes of Health (NIH) is the nation’s medical research agency whose mission is to enhance health and reduce the burden of illness and disability. To achieve this mission, the NIH has created a peer review system to identify the most promising biomedical research.

The peer review process includes two levels of review. The first level is by a Scientific Review Group (SRG) and the second by a National Advisory Council or Board of an Institute or Center. Only applications recommended for approval by both the SRG and the Council may be recommended for funding. Final decisions are made by the Director of the relevant NIH Institute or Center.

Despite efforts of NIH and other biomedical and behavioral research organizations to enhance the diversity of the workforce, more work remains to be done. Recent studies (Ginther et. al., 2011; 2012) have shown that African American researchers are less likely than White researchers to receive grant funding. These findings have raised concerns regarding the degree to which reviewers are demonstrating impartiality and fairness. Thus, CSR is seeking ideas to improve training practices for reviewers to enhance impartiality and fairness in peer review of grant applications. See Federal Register Notice (FRN Doc.2014-10203) for more details.

CHALLENGE: The Center for Scientific Review (CSR) is offering a prize for the best idea for Strategies to Strengthen Fairness and Impartiality in peer review of grant applications. Each idea should be provided in sufficient detail to assess its ability to address fairness and impartiality with regards to gender, race/ethnicity, institutional affiliation, area of science, and/or amount of research experience of applicants.

AWARDS: The prize amounts are a First prize of $10,000 and a Second prize of $5,000.


Submission Period: May 5, 2014 through 11:59pm (EST) on June 30, 2014

Judging Period: July 16, 2014 through August 29, 2014

Winners announced: September 2, 2014


After CSR receives the submissions, they will be evaluated in a two-stage process: (1) Technical merit will be evaluated for each submission’s potential to enhance fairness and impartiality in peer review (High, Medium, Low potential impact) by a panel of experts in fields relevant to peer review and reviewer bias, and (2) High Impact submissions will be evaluated and rank-ordered based on the judging criteria (see judging criteria below) by a panel of judges comprised of federal employees who will recommend the winning entries. The final awards will be approved by the Director of the Center for Scientific Review.

The judging criteria are as follows:
  • Demonstrates general knowledge of peer review practices.
  • Grounded in the empirical literature.
  • Feasible for implementation with reviewers in the NIH peer review system.
  • The proposed methods could be delivered to reviewers in a variety of delivery formats, including an electronic format.
  • Demonstrates understanding of the training literature.
  • Moves theory to practice.
  • Provides evidence that supports the effectiveness of the approach in resulting in fair and unbiased peer review.

Registration and submission process: There are 3 steps you must follow.
  1. Review the Rules below and go to (FRN Doc.2014-10203) for more detailed information.
  2. Complete the Challenge coversheet with your name and contact information.
  3. Do not provide identifying information on your submission materials. The review of your submission will be anonymous. Submit your idea to CSRDiversityPeerRev@mail.nih.gov. You may not submit your idea on the challenge.gov website. If you do not wish to apply online, send your materials to Office of the Director, Attention: Denise McGarrell, Center for Scientific Review, 6701 Rockledge Drive, Suite 3030, Bethesda, Maryland 20892.

Proposal acceptance or submission period: Idea proposals will be accepted beginning May 5, 2014 through 11:59 pm (EST) on June 30, 2014. Incomplete submissions will not be considered. All entries must be written in English.

Eligibility: The Challenge is open to any Contestant, defined as an individual or group of U.S. citizens, U.S. Territory residents or permanent residents of the United States who are 18 years of age or older (signed permission of a parent or guardian required if under 18 years of age). A Contestant may also be a community-based organization, school (private/public, elementary-college/university), non-profit organization, faith-based institution, philanthropic organization, local government, or small business. Proposed ideas may be incorporated into a Request for Applications (RFA), Request for Proposals (RFP) or an implemented study, but an award of a prize does not guarantee the proposed idea will be implemented.

Number of entries: One entry may be submitted per proposal/idea, but there are no limits to the number of ideas that one proposer or group of proposers may submit. However, if multiple versions of the same proposal/idea are submitted, CSR reserves the right to read only one proposal and will not consider the other submissions.

Finalist Selection and Notification: The winners will be selected by a two-stage process. The decisions of the panel on all matters relating to the Challenge are final. CSR reserves the right to refuse any proposals that do not adhere to the rules or that would result in a violation of Federal law, regulation, or the policies of the DHHS, NIH or CSR.

For questions about this challenge, please contact Monica Basco, Center for Scientific Review, phone: 301-300-3839 or email: CSRDiversityPeerRev@mail.nih.gov.

Refer References:

Ginther DK et al. (2011). Race, ethnicity, and NIH research awards. Science, 333 (1015-1019).

Ginther DK, Haak LL, Schaffer WT, & Kington R. (2012). Are race, ethnicity, and medical school affiliation associated with NIH R01 type 1 award probability for physician investigators? Academic Medicine, 87 (11), 1516-1524.ences: