COVID-19 is an emerging, rapidly evolving situation.

Get the latest public health information from CDC: https://www.coronavirus.gov
Get the latest research information from NIH: https://www.covid19.nih.gov/

Coronavirus Guidance for NIH Applicants and Grantees: https://grants.nih.gov/grants/natural_disasters/corona-virus.htm

CSR Overview


Scope of Review Operations at CSR

 
applications icon
76%
CSR reviews 76% of NIH proposals.
 
study sections icon
>200
In over 200 chartered or recurring Study Sections.
 
budget icon
0.4%
CSR does this with less than 0.4% of the total NIH budget.
 
reviewers icon
>18,000
CSR engages more than 18,000 distinct reviewers.
 
meetings icon
1,400
CSR conducts almost 1,400 review meetings each year.
 
review officers icon
>240
CSR employs >240 scientific review officers.
 
 
CSR’s Mission: To see that NIH grant applications receive fair, independent, expert, and timely reviews – free from inappropriate influences – so that NIH can fund the most promising research.
 
 

CSR Also Participates in the Review of These Initiatives and Inter-agency Collaborations

 

National Institutes of Health
NIH All-of-Us Program Reviews, plus Other Transaction Authority Reviews

National Institutes of Health
Brain Research through Advancing Innovative Neurotechnologies (BRAIN)

National Institutes of Health
NIGMS Maximizing Investigators’ Research Awards (MIRA)

National Institutes of Health
Native American Research Centers for Health (NARCH)

Global Alliance for Chronic Disease
Global Alliance for Chronic Disease

National Institutes of Health
All Division of Program Coordination, Planning, and Strategic Initiatives (DPCPSI)/Common Fund review

National Institutes of Health
Many IC PARS and multi-IC RFAs

Food & Drug Administration
FDA/Tobacco

National Institutes of Health
Cancer Moonshot

National Institutes of Health
Alzheimer's Disease (AD)

National Institutes of Health
Office of Research on Women’s Health (ORWH) at NIH - Specialized Centers of Research Excellence (SCORE) on Sex Differences

National Institutes of Health
All Fogarty International Center Reviews

National Institutes of Health
All Office of the Director (OD/Office of Research Infrastructure Programs (ORIP) Reviews

NIH, NSF & DOE
NIH-NSF and NIH-DOE
All USA-China Reviews



Return to top

 

Ongoing Evaluations

 
 

Early Career Reviewer (ECR) Program

The Early Career Reviewer (ECR) Program was established in late 2011. The program enriches the pool of NIH reviewers and gives scientists peer review experience that they can apply in crafting competitive applications.

  • 5,008 individuals have received ECR training and have served on study sections.
  • 17% of ECRs who have served on study sections are from under-represented racial/ethnic groups.
  • 48% of ECRs are women.
  • 484 former ECRs have served as members of standing study sections or are currently members. The program began in late 2011, focuses on new faculty, and time to tenure is typically 7 years. Thus this small percentage being appointed members of panels indicates progress and we expect this to grow with time.
early career reviewers icon

Analysis to be performed by CSR:

  1. Analysis of success rates for R01s submitted by ECRs compared to early career researchers, matched for relevant demographic and career characteristics
  2. Demographic analysis: gender, race, ethnicity, geographic distribution,and institution type for both the ECRs in our database and the ECRs recruited to review
  3. Survey ECRs to identify areas of program improvement

 

How Does Use of the Zoom Platform Affect Review Meetings?

Although CSR had prior experience with a range of formats for review meetings, the COVID-19 pandemic forced a move to entirely remote review meetings. CSR held > 600 review meetings using the Zoom platform from March to August 2020. All review meetings will continue to be held remotely at least through March 2021.

Moving to entirely virtual review meetings has given us the opportunity to collect a large amount of data to assess the effect of the Zoom review format on review process.

In the summer of 2020, we surveyed:

  • 3,288 reviewers
  • 128 scientific review officers
  • 73 staff supporting review meetings

Performed quantitative analyses of meeting duration and roster composition (n = 143 meetings, matched to their in-person equivalent).

early career reviewers icon
 
  • In-person meeting
  • In-person meeting + video conference
  • Video assisted meeting
  • Virtual meeting
  • Telephone assisted meeting

Reviewers and SROs prefer in-person meetings but the majority found overall quality of review in Zoom meetings to be the same as or better than that in in-person meetings. Nearly half of all reviewers (46%) reported having a shorter attention span during a Zoom meeting versus in an in-person meeting and quantitative analyses also suggest that Zoom meetings tend to be longer.

Reviewer engagement will be a focal point as we continue to examine the use of Zoom for review meetings.

There was no substantial change in female or minority representation on rosters for meetings held by Zoom versus those held in person in previous review cycles. Career stage and proportion of new reviewers was also unaffected by the move to review by Zoom.

CSR is continuing evaluations of Zoom review meetings.

Detailed findings are available here



Return to top



meeting group photo

 


 

Evaluating Panel Quality in Review (ENQUIRE)

 

CSR ENQUIRE

  • Clusters of study sections are formed based on scientific topics (instead of CSR managerial units) for review via ENQUIRE
  • Systematic, data-driven, continuous process – ~20% of CSR study sections are evaluated each year, every study section evaluated every 5 years
  • Stakeholder input and involvement – external scientific community, extramural programs at NIH, CSR staff
  • Iterative approach - continuous refinement of the process based on experience
  • Critical to success – matching referral of applications and reviewer expertise to redefined scientific scope of study sections

Multiple Possible Actions Follow

  • Change in scientific guidelines
  • Merge study sections
  • Create new study sections
  • Redistribute areas across study sections
  • Add emerging areas of science
  • Eliminate study sections
 

Process Overview for Each Cluster of Study Sections

 
  • Cluster Formation
 
  • External Scientific Evaluation Panel
 
  • Internal Process Evaluation Panel
 
  • Approvals
 
  • Implementation by CSR

Clusters Evaluated via ENQUIRE

Initiated in 2019 restructured and new study sections in place October 2020:

  • Healthcare Delivery and Patient Outcomes
    9 study sections evaluated, resulting in 11 new or modified study sections
    View the report
  • Gastrointestinal, Renal, Endocrine Systems
    11 study sections evaluated, resulting in 10 new or modified study sections
    View the report
  • Cardiac, Vascular and Hematologic Sciences
    10 study sections evaluated, resulting in 8 new or modified study sections
    View the report
  • Functional/Cognitive Neuroscience
    12 study sections evaluated, resulting in 11 new or modified study sections
    View the report

Initiated in 2020 and in progress:

  • Molecular and Cellular Basic Sciences
    16 study sections being evaluated
  • Epidemiology and Population Sciences
    10 study sections being evaluated
  • Oncology
    11 study sections being evaluated


Return to top

 

Integrity of the Peer Review Process

peer review group photo

Critically important for all of us

We must maintain the public trust in the NIH’s stewardship of taxpayer dollars to support U.S. biomedical science research.

Confidentiality is critical for candor in discussion and evaluation, and thus impacts the very basis of the peer review process.

Ensuring integrity requires the support of the entire research community – investigators, reviewers, study section chairs, NIH staff, institutional officials.

NIH is taking this issue very seriously. There do not appear to be widespread problems, but increased reporting and action is a cultural change.

What is the NIH Doing? Reporting & Action

We follow up on every allegation.

Cases are referred to Office of Management Assessment (OMA) – independent of CSR - an investigative unit conducts fact-finding, investigations, and issues a report of findings.

Actions have included

  • withdrawal of application
  • removal of reviewers from peer review committees
  • notification of the investigator/reviewer’s institution, which has led to personnel actions
  • pursuing government-wide suspension and disbarment
  • referral to other agencies for criminal violations

Making Reporting Easier

To make reporting easier, all scientific review officers include reporting information in their signature line in email messages:

Integrity matters. Say something! For concerns or questions about possible violations of peer review integrity, please contact your Scientific Review Officer, or the CSR Review Integrity Officer at csrrio@mail.nih.gov, or the NIH Review Policy Officer at reviewpolicyofficer@mail.nih.gov. See the NIH Guide Notice on integrity in review.

Scientific Review Officers regularly discuss integrity and reporting avenues. This is done through pre-meeting training of all reviewers and in opening statements at review meetings.



Return to top