FDA Advancing Innovation Through Deep Collaboration

Today, the FDA announced the beta release of precisionFDA, a community platform for NGS assay evaluation and regulatory science exploration. They are now accepting applications to the precisionFDA community, you can learn more and request access here.

Over the years, genetic testing has become increasingly useful in the diagnosis and treatment of disease in a few areas, including cancer, birth defects and rare diseases. However, for the majority of the population, precision medicine is still far from becoming a reality. FDA’s Center for Devices and Radiological Health and the agency’s Chief Health Informatics Officer, Dr. Taha Kass-Hout, embarked on a bold, new initiative for realizing precision medicine, by establishing precisionFDA. You can read more about their endeavor here.

Currently, most diagnostic tests follow a “one test-one disease paradigm” for evaluating analytical performance. However, diagnostic tests employing next-generation sequencing (NGS) technology can scan up to the entire genome, producing a massive amount of data and are capable of potentially detecting multiple conditions in a single test. The FDA realized that due to the advances in NGS-based technology, a new approach would be required for evaluating the accuracy of a test.

PrecisionFDA was established to help advance the regulatory science needed to assess the accuracy of genome tests and software. By providing a secure cloud-based platform that is open and transparent to the genomics community, researchers and test developers can explore NGS methodologies in order to spur innovation needed to develop necessary standards.

PrecisionFDA is a research sandbox that provides the genomics community with a web portal where they can experiment, share data and tools, collaborate, and define standards for evaluating analytical pipelines. Requirements for the precisionFDA platform were based on suggestions received through a public forum as well as use cases the FDA has gathered throughout the years.

Here are some key features of precisionFDA:

  • FILES – Upload your own files on cloud storage or generate files through running apps. You can publish reference data or any other files, or browse other members’ contributions.
  • APPS – Run mapping & variation calling pipelines or other Linux-based software apps on the cloud. Contribute your own software and scripts and let others explore them.
  • COMPARISONS – Quantify the similarity between two sets of genomic variants (VCF files). Compare your own test set (for a given biospecimen) to establish benchmark sets.
  • NOTES – Write and publish rich notes describing your work. Attach any files, comparisons or apps to your notes. Read what others are reporting and reproduce their workflows.

The concept of comparing two sets of variants (VCF files) is central to the exploration of regulatory science, and to the evaluation of NGS assays. The problem of comparing VCF files constitutes an active area of research. The precisionFDA building crew is represented in the Global Alliance for Genomics and Health (GA4GH) Benchmarking Task Team, which is expected (within the next year) to provide recommendations and/or software solutions for comparing VCFs and for counting, classifying, and reporting results. In the meantime, precisionFDA offers an initial VCF comparison framework, put together in consultation with NIST.

Check out the precisionFDA documentation for some great ideas for using comparisons, including assessing reproducibility and accuracy of NGS tests and bioinformatics variation calling pipelines.

PrecisionFDA follows a robust, audited set of policies, processes, and controls for security and compliance. When your data is in your private area, it is indeed private. It’s not visible to the FDA, members of the precisionFDA community, or any other entity. The platform provides users with access controls for their artifacts (files, apps, jobs, app assets, comparisons, and notes), so that they can either remain private, or published to the precisionFDA community.

Lastly, precisionFDA would be nothing without the support and engagement from its community members. As of today, early adopters have already contributed many valuable tools and reference datasets to the platform, and there are many more in the works! Here is a preview of what you can find on precisionFDA today:

  • NA12878 benchmark calls made by NIST (Genome in a Bottle v2.19)
  • NA12878 benchmark calls made by Illumina (Platinum Genome v8.0.1 and an updated kmer-filtered v7.1.0)
  • HuRef (J. Craig Venter) benchmark calls made by Roche/Bina
  • NA12878 exome test calls made by the Broad Institute
  • NA12878 whole-genome sequencing and test calls made by the Garvan Institute (using the Illumina HiSeq X Ten)
  • Software assets and apps for simulation and evaluation using VarSim, added by Roche
  • An app for local ancestry analysis with RFMIX, added by Stanford

Additional early members of the precisionFDA community:

  • 23andMe
  • Baylor College of Medicine
  • Counsyl
  • Emory Genetics Laboratory
  • GeneDX
  • Human Longevity Institute
  • Intel
  • Natera
  • Personalis
  • SeraCare

Above everything else, precisionFDA is a community, where people can collaborate, communicate, and even argue for the future of precision medicine. We are privileged to have been selected as the contractor for this pilot, and look forward to our collaboration with the FDA as the platform and community evolves. At this time, the precisionFDA platform includes features such as App Forking, Item Tracking, and Notes, which ignite collaboration, content expansion, and workflow validation and reproduction.

The Notes section, in particular, lets participants write and publish rich notes describing their thoughts and their work; for example, they can discuss how they used files, comparisons, and apps—which they can also attach to the note—to prove a certain point or to document a procedure. Community members can read what others have reported and access their attachments to take a closer look at that work or even reproduce it on their own.

We believe this new level of collaboration and reporting, together with everything else that precisionFDA has to offer, will define new frontiers for people to showcase to the FDA and to the rest of the community how to address the challenges of precision medicine in the 21st century.