FDA Advancing Innovation Through Deep Collaboration

Today, the FDA announced the beta release of precisionFDA, a community platform for NGS assay evaluation and regulatory science exploration. They are now accepting applications to the precisionFDA community, you can learn more and request access here.

Over the years, genetic testing has become increasingly useful in the diagnosis and treatment of disease in a few areas, including cancer, birth defects and rare diseases. However, for the majority of the population, precision medicine is still far from becoming a reality. FDA’s Center for Devices and Radiological Health and the agency’s Chief Health Informatics Officer, Dr. Taha Kass-Hout, embarked on a bold, new initiative for realizing precision medicine, by establishing precisionFDA. You can read more about their endeavor here.

Currently, most diagnostic tests follow a “one test-one disease paradigm” for evaluating analytical performance. However, diagnostic tests employing next-generation sequencing (NGS) technology can scan up to the entire genome, producing a massive amount of data and are capable of potentially detecting multiple conditions in a single test. The FDA realized that due to the advances in NGS-based technology, a new approach would be required for evaluating the accuracy of a test.

PrecisionFDA was established to help advance the regulatory science needed to assess the accuracy of genome tests and software. By providing a secure cloud-based platform that is open and transparent to the genomics community, researchers and test developers can explore NGS methodologies in order to spur innovation needed to develop necessary standards.

PrecisionFDA is a research sandbox that provides the genomics community with a web portal where they can experiment, share data and tools, collaborate, and define standards for evaluating analytical pipelines. Requirements for the precisionFDA platform were based on suggestions received through a public forum as well as use cases the FDA has gathered throughout the years.

Here are some key features of precisionFDA:

  • FILES – Upload your own files on cloud storage or generate files through running apps. You can publish reference data or any other files, or browse other members’ contributions.
  • APPS – Run mapping & variation calling pipelines or other Linux-based software apps on the cloud. Contribute your own software and scripts and let others explore them.
  • COMPARISONS – Quantify the similarity between two sets of genomic variants (VCF files). Compare your own test set (for a given biospecimen) to establish benchmark sets.
  • NOTES – Write and publish rich notes describing your work. Attach any files, comparisons or apps to your notes. Read what others are reporting and reproduce their workflows.

The concept of comparing two sets of variants (VCF files) is central to the exploration of regulatory science, and to the evaluation of NGS assays. The problem of comparing VCF files constitutes an active area of research. The precisionFDA building crew is represented in the Global Alliance for Genomics and Health (GA4GH) Benchmarking Task Team, which is expected (within the next year) to provide recommendations and/or software solutions for comparing VCFs and for counting, classifying, and reporting results. In the meantime, precisionFDA offers an initial VCF comparison framework, put together in consultation with NIST.

Check out the precisionFDA documentation for some great ideas for using comparisons, including assessing reproducibility and accuracy of NGS tests and bioinformatics variation calling pipelines.

PrecisionFDA follows a robust, audited set of policies, processes, and controls for security and compliance. When your data is in your private area, it is indeed private. It’s not visible to the FDA, members of the precisionFDA community, or any other entity. The platform provides users with access controls for their artifacts (files, apps, jobs, app assets, comparisons, and notes), so that they can either remain private, or published to the precisionFDA community.

Lastly, precisionFDA would be nothing without the support and engagement from its community members. As of today, early adopters have already contributed many valuable tools and reference datasets to the platform, and there are many more in the works! Here is a preview of what you can find on precisionFDA today:

  • NA12878 benchmark calls made by NIST (Genome in a Bottle v2.19)
  • NA12878 benchmark calls made by Illumina (Platinum Genome v8.0.1 and an updated kmer-filtered v7.1.0)
  • HuRef (J. Craig Venter) benchmark calls made by Roche/Bina
  • NA12878 exome test calls made by the Broad Institute
  • NA12878 whole-genome sequencing and test calls made by the Garvan Institute (using the Illumina HiSeq X Ten)
  • Software assets and apps for simulation and evaluation using VarSim, added by Roche
  • An app for local ancestry analysis with RFMIX, added by Stanford

Additional early members of the precisionFDA community:

  • 23andMe
  • Baylor College of Medicine
  • Counsyl
  • Emory Genetics Laboratory
  • GeneDX
  • Human Longevity Institute
  • Intel
  • Natera
  • NIST/GIAB
  • Personalis
  • SeraCare

Above everything else, precisionFDA is a community, where people can collaborate, communicate, and even argue for the future of precision medicine. We are privileged to have been selected as the contractor for this pilot, and look forward to our collaboration with the FDA as the platform and community evolves. At this time, the precisionFDA platform includes features such as App Forking, Item Tracking, and Notes, which ignite collaboration, content expansion, and workflow validation and reproduction.

The Notes section, in particular, lets participants write and publish rich notes describing their thoughts and their work; for example, they can discuss how they used files, comparisons, and apps—which they can also attach to the note—to prove a certain point or to document a procedure. Community members can read what others have reported and access their attachments to take a closer look at that work or even reproduce it on their own.

We believe this new level of collaboration and reporting, together with everything else that precisionFDA has to offer, will define new frontiers for people to showcase to the FDA and to the rest of the community how to address the challenges of precision medicine in the 21st century.

precisionFDA: A Community Approach for Submitting & Evaluating Diagnostic Tests, Powered by DNAnexus

DNAnexus has been awarded a research and development contract by the FDA’s Office of Health Informatics to build precisionFDA, an open source platform for community sharing of genomic information.

precisionFDA is a new approach for evaluating bioinformatics workflows, and is an integral part of FDA’s work in better understanding diagnostic tests that incorporate next-generation sequencing (NGS) technologies. As a component of the White House’s Precision Medicine Initiative, precisionFDA’s streamlined approach to evaluating NGS-based diagnostics and creation of reference datasets will build a community around best-practices resources and democratize the submission process to the FDA.

The FDA has adopted a community approach to crowd source reference analytical pipelines and datasets for the testing validation process by the community members who will be utilizing them. The DNAnexus Platform will deliver precisionFDA, providing the underlying cloud-based compute and data management infrastructure. In addition, DNAnexus will work with the FDA to build a community around its informatics platform to help drive standards around secondary analysis, the process of mapping, alignment, and variant calling of DNA sequence data.

The value of secondary analysis is undermined when datasets and bioinformatics tools are not harmonized for comparison and reproducibility. precisionFDA, with the help of the genomics community will streamline the process for submitting and validating NGS-based tests. Standardization will improve the evaluation process through consistent data quality, increased integration and reproducibility, and improved data exchange with collaborators.

Key objectives for precisionFDA include:

  • Exploring the use of a cloud-based portal, precisionFDA, to create a community around open-source genomic analysis pipelines, reference data, and analytical processing resources.
  • Determining appropriate and auditable levels of security, privacy, and governance control to ensure the protection of collaborators’ intellectual property and protected information, while enabling interaction within the community.
  • Providing an initial set of reference genomic data models and reference analysis pipelines.
  • Independent genomic analysis and data management work areas that can be kept private or shared with owner’s choice of collaborators, the public, or FDA for vetting or validation .

As a cloud-based informatics platform, precisionFDA will provide open source reference applications, reference datasets, and cloud-based compute and data management resources for the validation of NGS-based tests.

precisionFDA DNAnexus

precisionFDA is slated to provide test developers a flexible method for independently evaluating the accuracy and reproducibility of NGS analysis workflows, and to securely share results with collaborators and the FDA. DNAnexus expects the platform to be used broadly by NGS-based test providers, standards-making bodies, pharmaceutical and biotechnology companies, health care providers, academic medical centers, research consortia, and patient advocacy groups.

We predict that this new model for evaluating NGS-based tests will open up the process to a broader range of community members, who will benefit from open source reference data and applications and pay-for-use compute and storage resources, leveling the playing field for smaller test developers.

We are pleased to share this new and strategically important FDA initiative and look forward to collaborating with the genomics community in shaping this next evolution of precision medicine.