Translation Together Piloting HTS Reproducibility Study

Published 26 April 2019

Technology platform Small Molecules

 

A foundational element of the scientific method is reproducibility. It determines among others the reliability of an experiment and its potential to translate from preclinical research to clinical trials. However, the field is facing alarmingly high irreproducibility rates with up to 80% of published results not being reproducible. This is a major concern for researchers worldwide. (1)

What could be the cause of this?

Sources of irreproducible results have been attributed to four major categories:

  • biological reagents and reference materials,
  • study design,
  • data analysis and reporting,
  • and laboratory protocols.

Currently, several initiatives are aimed at addressing these issues to improve reproducibility in foundational research laboratories. However, to our knowledge, the reproducibility of preclinical experiments typically conducted by translational scientists has never been systematically investigated, such as high-throughput screening (HTS) of small molecule libraries.

 

A novel approach: Assessing the reproducibility of Small Molecules with High-Throughput Screening

The National Center for Advancing Translational Sciences (NCATS) and the Institute of Molecular and Translational Medicine (IMTM) are now piloting the first-ever HTS reproducibility study. This takes place under the auspices of Translation Together: a unique collaboration of leading translational research organizations from around the world.(2) Study participants utilize the Library of Pharmacologically Active Compounds for their experiments. This is a well-established database that is intentionally blinded. They are performing quantitative HTS against two robust luciferase biochemical assays with enzymes from Photinus pyralis and Renilla reniformis.

All experiments are executed with identical reagents and protocols supplied by NCATS. The participants will provide NCATS and the Institute for Molecular Medicine Finland (FIMM) with the raw data before the library compounds are un-blinded. Ultimately, all data analyses will be independently performed by NCATS, FIMM and each participant individually.

Although the assays are known to be robust, several other factors could affect reproducibility, including instrumentation, consumables and data analysis procedures. This important study has been carefully designed to identify sources of variation among participating centers and the results will be widely disseminated to improve the reliability of translational research practices.

 

The participants are engaged and actively discussing future studies intended to build on the current findings and address other aspects of reproducibility. Collaboration among these discovery centers has built strong and productive relationships with a unique ability to examine aspects of translational science that would not be otherwise possible.

This HTS reproducibility study is part of the EATRIS Quality Initiative. Among the participating institutions and pharma companies are five EATRIS members of the small molecules platform:

  1. National Center for Advancing Translational Sciences (NCATS), Rockville, USA
  2. Institute of Molecular and Translational Medicine (IMTM), Olomouc, Czech Republic
  3. Eli Lilly and Company, Indianapolis, USA
  4. Pfizer Inc., Groton, USA
  5. Flinders Centre for Innovation in Cancer, Flinders University, Bedford Park, Australia
  6. CNCCS- IRBM Science Park, Rome, Italy
  7. Fraunhofer Institute for Molecular Biology and Applied Ecology (IME), Hamburg, Germany
  8. Institute for Molecular Medicine Finland (FIMM), Helsinki, Finland
  9. Centre for Molecular Medicine Norway (NCMM), University of Oslo, Oslo, Norway
  10. LifeArc, Stevenage, UK
  11. Walter and Eliza Hall Institute of Medical Research, Parkville, Australia
  12. Istituto Nazionale Tumori – IRCCS – Fondazione G. Pascale, Napels, Italy

 

  1. Freedman L. P., Cockburn I. M., Simcoe T. S. The economics of reproducibility in preclinical research. PLoS biology. 2015 Jun 9;13(6):e1002165.
  2. Ussi, A. E., Kort, M., Coussens, N. P., Aittokallio, T. and Hajduch, M., 2017. In Search of System‐Wide Productivity Gains‐The Role of Global Collaborations in Preclinical Translation. Clin Transl Sci., 10(6), pp.423-425.