Skip to content

Irreproducible PreClinical Research Estimated to Cost $28 Billion/Year in US

2015-06-11

A new paper in PLOS Biology discusses the topic of non-reproducible research.  First, an estimate of the total cost of preclinical research in the United States:

“Extrapolating from 2012 data, an estimated US$114.8B in the United States [ref. 18] is spent annually on life sciences research, with the pharmaceutical industry being the largest funder at 61.8%, followed by the federal government (31.5%), nonprofits (3.8%), and academia (3.0%) [ref. 20]. Of this amount, an estimated US$56.4B (49%) is spent on preclinical research, with government sources providing the majority of funding (roughly US$38B) ”

Second, an estimate of irreproducible research is about 50%, based on a half-dozen studies with rates ranging from 51% to 88%, with 50-54% being the most likely:

” Using a conservative cumulative irreproducibility rate of 50% means that approximately US$28B/year is spent on research that cannot be replicated [see fig. 2]. Of course, uncertainty remains about the precise magnitude of the direct economic costs—the conservative probability bounds approach reported above suggest that these costs could plausibly be much smaller or much larger than US$28B.”

Third, academic research in drug development is typically repeated by the pharmaceutical industry before clinical research can begin:

” Academic research studies with potential clinical applications are typically replicated within the pharmaceutical industry before clinical studies are begun, with each study replication requiring between 3 and 24 months and between US$500,000 to US$2,000,000 investment [ref. 23]”

Improved reproducibility would improve the pharmaceutical industry’s return on investment by allowing clinical research to follow a larger percentage of preclinical studies, meaning more positive studies would result in eventual drug development.

The first aspect of improvement would involve adoption of standards for conduct of research and reporting of results:

” … one issue that has shown demonstrable impact on similar challenges in other settings is the expanded development and adoption of standards and best practices [ref. 13].”

The second improvement would be in training of scientists to include these best practices in their research.  The PLOS paper gives a list which provides a framework for improvement in standards and training, including the allocation of funding for these practices.

Here is an example of a practice that is not followed that could save money by reducing false results in the use of cancer cell lines:

“An illustrative example is the use and misuse of cancer cell lines. The history of cell lines used in biomedical research is riddled with misidentification and cross-contamination events [ref. 29], which have been estimated to range from 15% to 36% [ref. 30]. Yet despite the availability of the short tandem repeat (STR) analysis as an accepted standard to authenticate cell lines, and its relatively low cost (approximately US$200 per assay), only one-third of labs typically test their cell lines for identity [ref. 31].”

The process of validating reagents to prevent misidentification and cross-contamination, and the identification of cell lines to likewise avoid contamination and wrong identification, would eliminate a large proportion of primary errors in the performance of research, but it will be necessary to train, fund, and regulate in order to establish these basic procedures among the entire research community.  An outline of these procedures for training and so on is given as part of the conclusion of this paper.

This type of development is important for reducing errors and making research more reproducible.  Whether this can be done depends on the motivation of the groups that have the most investments in making research accurate: the funding bodies and the administrations of the institutions that do research.  Public awareness of the need for accuracy and reproducibility will improve the motivations of these groups.  Everyone wants to save money by doing things in the most efficient manner possible; what is required is the motivation to do the right thing from the beginning.

Resistance to changes of this nature can be expected from some individuals, particularly those who have been short-cutting the process of doing good research and those who are used to the situation as it now stands.  Those who want to read good research and those who want to do good research will be forced to put their livelihoods on the line to require improvements in research practices.

Here’s a link to the paper: http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002165

 

 

Advertisements
No comments yet

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: