Premise, Rigor, Sex as a Biological Variable

Does the NIH policy on rigor and transparency apply to all types of NIH grants?

The rigor and transparency policy applies to most research grant and mentored career development award applications submitted for the January 25, 2016 due date and beyond. Specific exceptions are listed in the research (NOT-OD-16-011) and in the activity codes list for rigor and transparency.

See NOT-OD-20-033 for details on implementation of the rigor and transparency policy for institutional training grants (Ts), institutional career development grants (K12 and KL2), and individual fellowships (Fs).

Funding opportunity announcements for which rigor and transparency should be addressed explicitly will include specific review language for rigor and transparency.

Where in grant applications should applicants address the four focus areas of the NIH policy on rigor and transparency?

Please see the Office of Extramural Research resource Guidance: Rigor and Reproducibility in Grant Applications for the most current information.

Rigor of the prior research, scientific rigor in the experimental design, and relevant biological variables such as sex should be addressed within the Research Strategy of research applications, as these elements are integral to the research plan. Because rigor of the prior research will be reviewed and scored as part of the Significance review criterion for research grant applications, applicants should address this as part of their corresponding Significance section in the Research Strategy. Scientific rigor and relevant biological variables will be reviewed and scored as part of the Approach review criterion.

For mentored career development award applications, all three areas (rigor of prior research, scientific rigor, and relevant biological variables such as sex) should be addressed in the Research Strategy and all three areas will be reviewed as part of the Research Plan.

Authentication of key resources will be addressed in a separate attachment and will not be scored.

What is the difference between "rigor of the prior research" and "significance"?

The rigor of the prior research will be reviewed as part of the Significance criterion for research grant applications. Plans to address weaknesses in the rigor of the prior research will be reviewed as part of the Approach criterion for research grant applications. Instructions for significance already include consideration of the importance of the problem, critical barriers to progress, how the proposed project will improve scientific knowledge, and how the field will change if the aims are achieved. Rigor of the prior research includes a retrospective consideration of the foundation for the application, rather than a prospective analysis should the aims be achieved.

The rigor of relevant prior research is just part of the consideration of significance, which includes consideration of the importance of the problem, critical barriers to progress, how the proposed project will improve scientific knowledge, and how the field will change should the aims are achieved.

Should preliminary data presented within the application conform to the updated instructions for rigor and transparency? Will I be expected to discuss in my application the strengths and weaknesses of my own preliminary data as part of the scientific premise? What if a publication I cite does not include elements of rigor and transparency?

Prior research cited by an investigator as key support for the proposed project should be assessed for strengths and weaknesses in rigor, regardless of whether it is the investigator’s own preliminary data (published or unpublished) or published data from others. Any weaknesses or gaps in rigor, or reporting on rigor, must be acknowledged under Significance. Plan to address weaknesses or gaps in the rigor of the prior research must be described under the Approach. Preliminary data that serve as proof of concept for a technique and/or expertise do not need to be assessed.

In exploratory/developmental grant applications and other activities for which preliminary data are not required, how should the rigor of the prior research addressed?

The instructions for rigor of the prior research (under Significance and Approach, for research grant application) relate to the scientific rigor of the foundation of knowledge upon which the research questions are posed, including preliminary data if they are provided. If preliminary data are not provided in an application, a critical assessment of the scientific literature that supports and/or contradicts the research question(s) ought to be provided.

What does "scientific rigor" mean?

Scientific rigor is the strict application of the scientific method to ensure robust and unbiased experimental design, methodology, analysis, interpretation and reporting of results. This includes full transparency in reporting experimental details so that others may reproduce and extend the findings. Investigators should apply the elements of rigor that are appropriate for their science.

What does it mean to consider sex as a biological variable?

Appropriate strategies that consider sex as a biological variable (SABV) depend on the context of the research question, existing knowledge about male and female biology and behavior in a given area of research, available methodology, and other factors. NIH review committees will evaluate consideration of sex as a biological variable as part of their assessment of the research strategy for applications for research involving vertebrate animals and humans. An SABV decision tree for reviewers is now available and is also useful for applicants. Accounting for sex as a biological variable in applications for NIH-funded research could be reflected in:

  • relevant review of available literature on the influence of biological sex
  • formulation of research questions
  • incorporating both males and females into studies
  • articulating strong justification for a single-sex study
  • consideration of the influence of sex in study design
  • stratified randomization of males and females into experimental conditions
  • characterization of study results for males and females
  • examination of treatment or toxicity effects for each sex separately
  • consideration of the influence of sex in the interpretation of study results
  • appropriate generalization of research findings
  • Note that this is not an exhaustive list. Please see Clayton, 2015 for additional information on what it means to consider sex as a biological variable.

Which relevant biological variables do we need to consider?

Applicants should consider the biological variables that are relevant to the experimental design of the study. The choice of animal model or human population to be included will vary with the scientific topic of the proposed research. For example, sex, age, weight, and underlying health conditions are biological variables that may affect outcome and should be considered where applicable.

In particular, sex is a biological variable that has been frequently ignored in animal study designs and analyses, leading to an incomplete understanding of potential sex-based differences in basic biological function, disease processes, and treatment responses. NIH expects that sex as a biological variable (SABV) will be considered in research designs, analyses, and reporting of vertebrate animal and human studies. Strong justification from the scientific literature, preliminary data, or other relevant considerations must be provided for applications proposing to study only one sex.

Does the sex of primary cells or tissue explants, (i.e. cells or tissues removed directly from the animal or human) need to be accounted for under the policy to Consider Sex as a Biological Variable?

Applicants are instructed to discuss their consideration of sex as a biological variable for vertebrate animal and human studies. Sex should be accounted for when using primary cells, tissue explants, or other samples taken directly from the animal or human. In instances where cells or tissues will be implanted into a host (e.g. patient-derived xenografts), the sex of both the sample and the host should be considered and reported. A strong scientific justification must be provided for applications proposing the use of a single sex.

Does the Consideration of Sex as a Biological Variable policy (NOT-OD-15-102) include established cells lines?

It is important that researchers using established cell lines consider the possible role of sex in their research. However, NIH recognizes current challenges to the authentication of the sex of established cell lines. At this time, NIH is working to enhance strategies and techniques to address these challenges. Although established cell lines are not explicitly addressed in this notice, NIH encourages researchers to consider sex as a biological variable and be transparent in reporting the sex of established cell lines, when known, as well as any SABV-related study results.

How should applicants address scientific rigor and sex as a biological variable when the research involves scarce animal resources?

Applicants must provide strong justification to study only one sex. Scarce animal resources may be considered adequate justification, based on the evidence of scarcity. Acceptable justifications may also include the study of sex-specific conditions or phenomena (e.g., ovarian or prostate cancer), or investigations in which the study of one sex is scientifically appropriate.

The absence of evidence regarding sex differences in an area of research does not constitute strong justification to study only one sex. Cost also is not a consideration in determining whether both sexes are to be included in experiments.

Applicants should provide a justification that the species are appropriate for the proposed research in the vertebrate animals section. The rationale for the number of subjects planned for study should now be explained in the Research Strategy section, for research grant applications.

What are "key biological and/or chemical resources"?

The quality of resources used to conduct research is critical to the ability to reproduce the results.

Key resources refer to established resources that will be used in the proposed research. Key biological and/or chemical resources include, but are not limited to, cell lines, specialty chemicals, antibodies and other biologics. Key biological and/or chemical resources may or may not have been generated with NIH funds and:

  • May differ from laboratory to laboratory or over time;
  • May have qualities and/or qualifications that could influence the research data; and
  • Are integral to the proposed research.

Standard laboratory reagents that are not expected to vary do not need to be included in the plan. Examples are buffers and other common biologicals or chemicals.

Depending on the research study, biological samples may be considered key biological resources that need to be authenticated if they are an established resource, particularly if the investigator received the samples from an outside source.

Each investigator will have to determine which resources used in their research fit these criteria and are therefore key to the proposed research.

Where should I go if I have other questions about these issues?

Please visit the larger NIH Rigor FAQ web page.