Premise, Rigor, Sex as a Biological Variable

Does the NIH policy on rigor and transparency apply to all types of NIH grants?

The rigor and transparency policy applies to most research grant and mentored career development award applications submitted for the January 25, 2016 due date and beyond. Specific exceptions are listed in the research (NOT-OD-16-011) and in the activity codes list for rigor and transparency.

See NOT-OD-20-033 for details on implementation of the rigor and transparency policy for institutional training grants (Ts), institutional career development grants (K12 and KL2), and individual fellowships (Fs).

Funding opportunity announcements for which rigor and transparency should be addressed explicitly will include specific review language for rigor and transparency.

Where in grant applications should applicants address the four focus areas of the NIH policy on rigor and transparency?

Please see the Office of Extramural Research resource Guidance: Rigor and Reproducibility in Grant Applications for the most current information.

Rigor of the prior research, scientific rigor in the experimental design, and relevant biological variables such as sex should be addressed within the Research Strategy of research applications, as these elements are integral to the research plan. Because rigor of the prior research will be reviewed and scored as part of the Significance review criterion for research grant applications, applicants should address this as part of their corresponding Significance section in the Research Strategy. Scientific rigor and relevant biological variables will be reviewed and scored as part of the Approach review criterion.

For mentored career development award applications, all three areas (rigor of prior research, scientific rigor, and relevant biological variables such as sex) should be addressed in the Research Strategy and all three areas will be reviewed as part of the Research Plan.

Authentication of key resources will be addressed in a separate attachment and will not be scored.

I’m confused about the term “scientific premise or rigor” and how it is different from “significance.” Can you tell me what reviewers are supposed to be looking for?

The NIH is moving away from the term “scientific premise” for applications that come in for due dates of January 25, 2019, and beyond. In evaluating these applications, reviewers still will be asked whether the prior research that serves as the key support for the proposed project is rigorous.

Scientific premise or rigor is just part of the consideration of significance, which includes consideration of the importance of the problem, critical barriers to progress, how the proposed project will improve scientific knowledge, and how the field will change should the aims are achieved.

In exploratory/developmental grant applications and other activities for which preliminary data are not required, how should scientific premise and scientific rigor be addressed?

The instructions for scientific premise (under Significance, for research grant applications) relate to the scientific rigor of the foundation of knowledge upon which the research questions are posed, including preliminary data if they are provided; in contrast, the instructions for scientific rigor (under Approach, for research grant applications) relate to the rigor of the proposed research. If preliminary data are not provided in an application, a critical assessment of the scientific literature that supports and/or contradicts the research question(s) can be provided.

How will reviewers be instructed to assess applications to address the NIH policy on rigor and transparency?

Reviewers are directed to consider the strengths and weaknesses associated with each of the review criteria and to weigh them appropriately for the specific application assigned to them. Thus, the NIH policy on rigor and transparency will contribute to the relevant criterion scores and the overall impact score of each application, with the exception of Authentication of Key Biological and/or Chemical Resources.

For research grant applications, reviewers will be instructed to evaluate scientific premise as part of the significance criterion, and to evaluate scientific rigor and relevant biological variables such as sex as part of the approach criterion, as well as in their assessment of overall impact.

Reviewers will comment on the proposed Key Biological and/or Chemical Resource Authentication Plan as an additional review consideration, but should not consider it in the overall impact score.

Details on the updated application instructions, progress reporting, and review language can be found in the following guide notices:

Rigor and transparency in research grant applications (NOT-OD-16-011)

Rigor and transparency in career development awards (NOT-OD-16-012)

What does "scientific rigor" mean?

Scientific rigor is the strict application of the scientific method to ensure robust and unbiased experimental design, methodology, analysis, interpretation and reporting of results. This includes full transparency in reporting experimental details so that others may reproduce and extend the findings. Investigators should apply the elements of rigor that are appropriate for their science.

Does the sex of primary cells or tissue explants, (i.e. cells or tissues removed directly from the animal or human) need to be accounted for under the policy to Consider Sex as a Biological Variable?

Applicants are instructed to discuss their consideration of sex as a biological variable for vertebrate animal and human studies. Sex should be accounted for when using primary cells, tissue explants, or other samples taken directly from the animal or human. In instances where cells or tissues will be implanted into a host (e.g. patient-derived xenografts), the sex of both the sample and the host should be considered and reported. A strong scientific justification must be provided for applications proposing the use of a single sex.

Does the Consideration of Sex as a Biological Variable policy (NOT-OD-15-102) include established cells lines?

It is important that researchers using established cell lines consider the possible role of sex in their research. However, NIH recognizes current challenges to the authentication of the sex of established cell lines. At this time, NIH is working to enhance strategies and techniques to address these challenges. Although established cell lines are not explicitly addressed in this notice, NIH encourages researchers to consider sex as a biological variable and be transparent in reporting the sex of established cell lines, when known, as well as any SABV-related study results.

How should applicants address scientific rigor and sex as a biological variable when the research involves scarce animal resources?

Applicants must provide strong justification to study only one sex. Scarce animal resources may be considered adequate justification, based on the evidence of scarcity. Acceptable justifications may also include the study of sex-specific conditions or phenomena (e.g., ovarian or prostate cancer), or investigations in which the study of one sex is scientifically appropriate.

The absence of evidence regarding sex differences in an area of research does not constitute strong justification to study only one sex. Cost also is not a consideration in determining whether both sexes are to be included in experiments.

Applicants should provide a justification that the species are appropriate for the proposed research in the vertebrate animals section. The rationale for the number of subjects planned for study should now be explained in the Research Strategy section, for research grant applications.

What are "key biological and/or chemical resources"?

The quality of resources used to conduct research is critical to the ability to reproduce the results.

Key resources refer to established resources that will be used in the proposed research. Key biological and/or chemical resources include, but are not limited to, cell lines, specialty chemicals, antibodies and other biologics. Key biological and/or chemical resources may or may not have been generated with NIH funds and:

  • May differ from laboratory to laboratory or over time;
  • May have qualities and/or qualifications that could influence the research data; and
  • Are integral to the proposed research.

Standard laboratory reagents that are not expected to vary do not need to be included in the plan. Examples are buffers and other common biologicals or chemicals.

Depending on the research study, biological samples may be considered key biological resources that need to be authenticated if they are an established resource, particularly if the investigator received the samples from an outside source.

Each investigator will have to determine which resources used in their research fit these criteria and are therefore key to the proposed research.

Where should I go if I have other questions about these issues?

Please visit the larger NIH Rigor FAQ web page.