The Launch of precisionFDA Consistency Challenge
Last week the White House held a Precision Medicine Initiative (PMI) Summit, where government agencies discussed their progress on genomics-based personalized care, real people shared their success stories with precision medicine, and President Obama himself reiterated his vision for precision medicine. It was thrilling for DNAnexus to hear precisionFDA, the “online, cloud-based portal” that DNAnexus built for the FDA, based upon FDA requirements, specifically acknowledged (29:45-30:33). You can watch full coverage of the PMI Summit here.
It is no surprise that we have finally reached the era where cloud-based genomics is making it into White House announcements. The announcement also included the launch of the first precisionFDA consistency challenge, calling on members of the genomics community to assess their bioinformatics software on supplied reference human datasets. Called the precisionFDA Consistency Challenge, the goal is to engage genomics innovators to improve reproducibility and accuracy of next-generation sequencing (NGS) pipelines in order to achieve more consistent results from genetic tests and advance precision medicine.
PURPOSE OF THE CHALLENGE
Currently a single NGS test can identify genetic variants ranging from thousands to in the millions. The results, in some instances, are already being used to diagnose and treat disease. While NGS tests are currently used in clinical applications in oncology, non-invasive prenatal testing (NIPT), and rare disease, there is still room to improve consistency as they become more broadly adopted in clinical practice. A better understanding of accuracy and reliability of the results for specific NGS tests will help us get closer to personalized treatments and improve patient care.
Multiple sequencing technologies (Illumina XTen, Illumina HiSeq, PacBio RSII) can be used for human whole genome sequencing, and each has their own accuracy or reproducibility error profiles for different parts of the genome. The means to establish methodologies to assess the accuracy of a technology for specific variant types or regions across the genome would help advance evaluation of novel NGS-based tests for clinical applications. Unfortunately, there are still inconsistencies where an NGS-based test can report differing results. However, by establishing appropriate standards, inter-test variation can be minimized, allowing patients and physicians to place greater confidence in test results and the resulting treatments.
The FDA has delivered a new approach, precisionFDA, to help establish standards around secondary analysis – the process of mapping, aligning, and calling the variants of DNA sequence data. To jumpstart the engagement and improve techniques on the precisionFDA platform, the FDA’s first challenge is focused on consistency.
CHALLENGE DETAILS
The process of human WGS pipeline development and validation typically relies on mapping the sequenced reads to a well-known reference genome, then identifying the differences between the results and the reference dataset. Participants who join the challenge are asked to download two FDA-provided datasets (one contributed by the Garvan Institute of Medicine, and one by Human Longevity Inc. — both corresponding to sequencing of the well-characterized NA12878 sample), process it through their pipelines, upload results back to precisionFDA, and compare them to other files. The challenge provides a common frame of reference for measuring some of the aspects of reproducibility and accuracy of the participant’s pipeline.
WHO CAN PARTICIPATE?
The challenge is open to all innovators in the field of human genomics. If you’re not already a member of precisionFDA, you will need to request access to get started. You have until April 25th to submit your software assessments.
DETERMINING WINNERS*
Results will be ranked on the precisionFDA website for achievements in eight categories. See the Determining Winners section on the challenge webpage for full details. In addition to exclusive bragging rights, your results, comparisons, and methods will be featured on the precisionFDA platform highlighting your technical contributions.
DNAnexus is incredibly excited about this challenge – the idea of genomics innovators working together to advance quality standards is something that gets us fired up. As of this blog post, precisionFDA hosts more than 1000 community members on the platform representing nearly 500 organizations. We are proud to support this novel community-contribution model for evaluating bioinformatics pipelines to help address the challenges of precision medicine.
* Winning a precisionFDA category is an acknowledgement by the precisionFDA community and does not imply FDA endorsement of any organization, tool, software, etc.
