National Primate Research Centers Address Rigor and Reproducibility Challenges

In recent years, scientists have grown increasingly aware of the importance of conducting research that is both rigorous and reproducible. The need for increased rigor (robustness and lack of bias) and reproducibility (the ability to be repeated biologically, analytically, systemically, or conceptually) affects all areas of science, but approaches for ensuring scientific rigor differ among scientific disciplines.

Figure 1. Biomedical research models using nonhuman primates provide researchers insight into a wide array of genetic diseases in humans. Image courtesy of Marquess789/Shutterstock.

Dr. Roberts recently co-authored a review article¹ in the American Journal of Primatology addressing rigor and reproducibility in nonhuman primate (NHP) research. He is part of a team of representatives from the seven Office of Research Infrastructure Programs (ORIP)-supported NPRCs who contributed to the article. The article discusses the importance of transparency, evaluation, and collaboration in NHP research to address this issue.

The NPRC Consortium facilitates the effective use of NHPs by scientists engaged in biomedical research. The NPRCs complement and enable the missions of the other NIH Institutes and Centers by providing the animals, facilities, expertise, and resources required to enable NHP research in specific disease areas. Located in various parts of the United States, the NPRCs are available to investigators on a national basis.

ORIP is committed to promoting rigor and reproducibility across all types of animal models for biomedical research. Dr. Eliza Bliss-Moreau, an associate professor in the Department of Psychology at the University of California, Davis, and a core scientist at the California NPRC, emphasized that many of the recommendations from the review article can be applied to nearly all research models. The topic, however, is particularly important for NHPs for two reasons: (1) there are special ethical considerations associated with carrying out research with NHPs, and (2) NHP studies are significantly more resource-intensive than those utilizing other animal models, making them less likely to be replicated than those using other animal models. Therefore, research using NHPs must be rigorous to ensure that a given study generates the most robust data.

Figure 2. The need for increased rigor and reproducibility affects all areas of science but is particularly crucial in research involving nonhuman primates. Image courtesy of the Specific-Pathogen-Free Baboon Research Resource, The University of Texas MD Anderson Cancer Center.

Dr. Bliss-Moreau is first author of the American Journal of Primatology article. She explained that the NPRC team is working to encourage the adoption of best practices to increase research rigor and reproducibility among investigators that use NHPs in their research, across a wide variety of scientific disciplines. Most efforts to address rigor and reproducibility concerns have been focused on specific research areas. In the past, individual journals, scientific societies, or funding agencies have established guidelines to increase rigor and reproducibility in scientific research.

The NPRC team spans the various research areas within the NPRC Consortium, many of which have different accepted standards for experimental design, data analysis, and data sharing. Dr. Bliss-Moreau emphasized that the team’s unique composition allowed the incorporation of diverse perspectives that informed their recommendations for the NHP community as a whole.

“We got together to think about how the resources of the NPRCs could be leveraged to improve the science that people are doing on NHPs. The paper is the product of those discussions, and it represents points of synergy across disciplines,” Dr. Bliss-Moreau emphasized. “I think that’s important because it’s really an interdisciplinary approach … It represents views that should be useful regardless of a scientist’s specific research focus.”

Dr. Bliss-Moreau explained that a perception of the scientific community is that failures to reproduce results indicate that the science is of poor quality or lacking in rigor. In some cases, this may be true. In other cases, however, failures to reproduce results could be attributable to systematic differences in procedures. The challenge is that these two cases cannot be separated without transparency about methods, analytics choices, and interpretation. As a result, evaluations of rigor and the ability to reproduce results relate directly to transparency and openness in science—a key point of the article. Dr. Bliss-Moreau emphasized that this need for transparency spans the breadth of the NHP research community.

“All of the fields represented at the table are having these conversations within them, but this was a really unique opportunity to discuss these things across disciplines,” Dr. Bliss-Moreau stated. “Ethical advances with NHPs are really informed by scientific advances. Research ethics and science are both evolving processes, and I think this report reflects one snapshot of a perspective on how we can improve science. That perspective will continue to evolve.”

References

¹ Bliss-Moreau E, Amara RR, Buffalo EA, et al. Improving rigor and reproducibility in nonhuman primate research. Am J Primatol. 2032; e23331. doi: 10.1002/ajp.23331.