Bias Awareness and Mitigation Training for Reviewers, Chairs, and SROs

trophy icon

As of Dec 2022, 16,646 reviewers have completed the training.

identify icon

91% of reviewers thought that the training substantially improved their ability to identify bias in peer review.

bias icon

93% of reviewers stated the training made them substantially more comfortable intervening against bias.

                            Data for the Jan 2022 Advisory Council cycle.

CSR developed training specifically targeted toward mitigating the most common biases in the peer review process. The training includes personal testimonials, interactive exercises, and a narrated mock study section demonstrating techniques to intervene – all based on real-life examples. The training was developed with the assistance of a diverse CSR Advisory Council Working Group. Training has been provided to all CSR reviewers since August 2021.


Return to top


Reporting Avenues for Bias, Unfair Reviews, Uncivil Conduct on Panels


CSR launched a widely-publicized reporting avenue for issues related to respectful interactions, bias or anything else that could affect the fairness of the review process. The reporting channel is open to all – investigators, reviewers, and program staff.


Every allegation is carefully investigated by CSR senior management.



If we agree the review was biased/flawed, CSR will re-review application in the same council round. If we don’t agree, the official NIH appeals process remains available to all investigators.


Closure and Culture Change    
Re-review? CSR Scientific Division Director discusses the issue with the reviewer, if appropriate. Actions might also include not inviting the reviewer to serve in the future.


Report a Bias Incident    
Send a message to

Return to top    

Broadening the Reviewer Pool to Diversify Peer Review Groups

scientific background icon

Scientific Background






Career Stage


Peer Review Experience


There is a critical need for NIH to hear diverse perspectives to fulfill peer review’s mission of identifying the best, most novel science. The most effective, highest-quality review committees are broadly diverse in multiple dimensions including 1) scientific background and perspective; 2) demography; 3) geography; 4) career stage; 5) peer review experience. The value of diversity is evident in expectations of scientific review officers in recruiting for panels and in our investment of resources and public programs to more easily identify reviewers who are not already known to NIH.


Study section membership

The selection process for members of standing study sections is thorough and involves multiple levels of oversight and approval. Guidelines for staff in developing a slate of nominees explicitly addresses the value of diversity on panels and the advance planning process includes an analysis of current diversity on the panel, gaps, and plans to address those gaps

Currently, CSR is increasing focus on diversity of special emphasis panels (SEP) and has developed tools to allow SROs and supervisors to track diversity on SEPs.

Early Career Reviewer Program

CSR’s Early Career Reviewer (ECR) program offers early career scientists the opportunity to gain first-hand experience in peer review. In 2020, the program was expanded by requiring all standing study sections to include two ECRs at each meeting. This benefits ECRs by providing experience they can use in crafting their own grant applications and benefits CSR in that ECRs are more diverse. In 2021, 16.8% of ECRs were underrepresented minorities, compared to 10.3% for all CSR reviewers.

Development of tools to assist SROs in finding new, qualified reviewers


CSR has developed an internal database that allows scientific review officers (SRO) to more easily identify active scientists who might not already be known to NIH. The tool includes scientists funded through other government agencies and some non-profit organizations, those recommended by NIH staff, and those recommended by scientific societies.

CSR also regularly facilitates sharing among SROs of best practices in broader recruitment strategies.

Impact of CSR’s Efforts


Return to top


Exploring Changes to Review to Make it More Fair and Effective

Exploring blinded review processes photo

Leading efforts to simplify review criteria

CSR Advisory Council working groups (Clinical Trials, Non-Clinical Trials) developed recommendations to simplify review criteria in a way that focuses reviewers on the importance and feasibility of the research proposed and in which reputation of the investigator and institution, in the global sense, do not have a place. The reorganization of the current five review criteria into three factors allow for the possibility of a multi-stage, partially-blinded review process in the future. These recommendations were considered and modified by NIH leadership; see details of the proposed review framework. NIH gathered additional input from the scientific community on the proposed changes through a Request for Information, which closed March 10, 2023. An overview of the input received from more than 800 individuals, scientific societies, and academic institutions has been shared on Review Matters and a content analysis posted.


Improving NRSA Fellowship Review

A CSR Advisory Council Working Group on Fellowship Review developed recommendations to modify the review criteria and change the required application materials for NRSA fellowship applications. The changes are expected to make the process more fair and more effective in identifying the next generation of promising scientists.


Review criteria will be modified to better focus reviewers on three key assessments: (1) potential of the applicant; (2) strength of the science; (3) quality of the training plan. In order to provide equal opportunity, accomplishments of the applicant will be evaluated in the context of their opportunities. Sponsor and institution will be evaluated with respect to the quality of the proposed science and quality of the training plan, minimizing effects of reputation. Changes in required application materials fit with the modified review criteria. Importantly, coursework grades will no longer be considered.

Details of the recommended changes can be found here. These recommendations were considered and approved by NIH leadership. An RFI was issued to gather additional input from the public and closed in June 2023. Responses to the RFI were supportive but underscored the need for ample communications and thoughtful guidance and training for investigators, reviewers, and NIH staff. An NIH Guide Notice will be issued in late 2023 with more information.

Collaboration with the Common Fund High Risk, High Reward (tR01) program

  • CSR collaborated with the Common Fund to test a multi-stage review process in which information about the investigator and institution is provided after assessment of the abstract, aims, and research plan. Study section reviews took place in April 2021 and the process is being evaluated by an external contractor. Initial results are encouraging with a statistically significant increase in the demographic diversity of the applicant pool.
  • CSR developed and supports the TRA Anonymization Check, an online tool that allows applicants to verify that their specific aims and research strategy sections do not contain identifying information.

CSR investigated the effects of anonymization on review outcomes

CSR conducted a large-scale study of the effect of redacting identifying information from grant applications on the preliminary assessment of scientific and technical merit (Nakamura et al, 2021).

400 R01s from Black/African American (B/AA) principal investigators (PI), 400 from matched white PIs, 400 from randomly-selected white PIs
Full and redacted versions underwent simulated peer review
Data collection and analysis were performed by an external contractor using a preregistered plan



Redaction did not affect scores of B/AA PIs but worsened scores of white PIs (significant, but small effect size)
21% of the time, reviewers identified the PI despite redaction. Removing these instances did not change the findings.

What does this mean

Isolating the effect of race is challenging due to secondary, linked variables (e.g., institutional prestige, investigator pedigree) that are tied to racial disparities. Redaction may have reduced these “halo effects”.
Findings support review approaches that diminish the role of PI identity in review.

Return to top

Last updated: 12/20/2023 13:46