I hadn’t intended to write about precisionFDA going live – this post by Dr. Taha Kass-Hout and Elaine Johanson of the FDA provides a terrific summary, and this post by Angela Anderson of DNAnexus offers valuable additional context. However, I found myself today so excited by this project and what it represents that I can’t resist offering a few additional thoughts about what makes this initiative so special.
First, it addresses an important problem in the field: the analytic validity of NGS tests. The ready availability of relatively inexpensive sequencing has enabled us to contemplate diagnostic sequencing at a scale that would have been difficult to imagine even a decade ago. At the same time, the drive to apply sequencing in different clinical contexts raises a critically important question: do I trust this test? A key starting point for clinical interpretation of DNA data is to agree on the sequence itself. If your procedure and analysis reports that a particular sequence in a DNA sample is “GATCGATC” and my procedure and analysis of the same DNA says the sequence is “GATTGATC,” then we’ve got a problem. precisionFDA will allow users to compare approaches, figure out what’s working, and determine where refinements might be needed.
Second, precisionFDA represents a novel and forward-thinking approach to regulation. Rather than envisioning governmental regulators as the folks who will define and then impose a specific set of performance standards, precisionFDA instead sees the government as providing the platform that will enable the NGS community to evolve the standards on their own — organically and transparently.
Finally, the ability to design, refine, and deploy this platform in such a rapid and agile fashion reflects in part the value of well-conceptualized public-private partnerships, in this case between the FDA and DNAnexus. By intentionally leveraging the skills and capabilities of a company like ours, the FDA was able to implement and realize their exciting and ambitious vision.
The ultimate success of the precisionFDA platform will of course depend upon how well it serves the community it is intended to support. However, it’s hard to think of a more auspicious beginning, and my hope would be that success here will encourage more leaders to evaluate the potential of public/private partnerships to deploy platforms that leverage the power of a distributed innovation community to address important shared challenges.
Today, the FDA announced the beta release of precisionFDA, a community platform for NGS assay evaluation and regulatory science exploration. They are now accepting applications to the precisionFDA community, you can learn more and request access here.
Over the years, genetic testing has become increasingly useful in the diagnosis and treatment of disease in a few areas, including cancer, birth defects and rare diseases. However, for the majority of the population, precision medicine is still far from becoming a reality. FDA’s Center for Devices and Radiological Health and the agency’s Chief Health Informatics Officer, Dr. Taha Kass-Hout, embarked on a bold, new initiative for realizing precision medicine, by establishing precisionFDA. You can read more about their endeavor here.
Currently, most diagnostic tests follow a “one test-one disease paradigm” for evaluating analytical performance. However, diagnostic tests employing next-generation sequencing (NGS) technology can scan up to the entire genome, producing a massive amount of data and are capable of potentially detecting multiple conditions in a single test. The FDA realized that due to the advances in NGS-based technology, a new approach would be required for evaluating the accuracy of a test.
PrecisionFDA was established to help advance the regulatory science needed to assess the accuracy of genome tests and software. By providing a secure cloud-based platform that is open and transparent to the genomics community, researchers and test developers can explore NGS methodologies in order to spur innovation needed to develop necessary standards.
PrecisionFDA is a research sandbox that provides the genomics community with a web portal where they can experiment, share data and tools, collaborate, and define standards for evaluating analytical pipelines. Requirements for the precisionFDA platform were based on suggestions received through a public forum as well as use cases the FDA has gathered throughout the years.
Here are some key features of precisionFDA:
FILES – Upload your own files on cloud storage or generate files through running apps. You can publish reference data or any other files, or browse other members’ contributions.
APPS – Run mapping & variation calling pipelines or other Linux-based software apps on the cloud. Contribute your own software and scripts and let others explore them.
COMPARISONS – Quantify the similarity between two sets of genomic variants (VCF files). Compare your own test set (for a given biospecimen) to establish benchmark sets.
NOTES – Write and publish rich notes describing your work. Attach any files, comparisons or apps to your notes. Read what others are reporting and reproduce their workflows.
The concept of comparing two sets of variants (VCF files) is central to the exploration of regulatory science, and to the evaluation of NGS assays. The problem of comparing VCF files constitutes an active area of research. The precisionFDA building crew is represented in the Global Alliance for Genomics and Health (GA4GH) Benchmarking Task Team, which is expected (within the next year) to provide recommendations and/or software solutions for comparing VCFs and for counting, classifying, and reporting results. In the meantime, precisionFDA offers an initial VCF comparison framework, put together in consultation with NIST.
Check out the precisionFDA documentation for some great ideas for using comparisons, including assessing reproducibility and accuracy of NGS tests and bioinformatics variation calling pipelines.
PrecisionFDA follows a robust, audited set of policies, processes, and controls for security and compliance. When your data is in your private area, it is indeed private. It’s not visible to the FDA, members of the precisionFDA community, or any other entity. The platform provides users with access controls for their artifacts (files, apps, jobs, app assets, comparisons, and notes), so that they can either remain private, or published to the precisionFDA community.
Lastly, precisionFDA would be nothing without the support and engagement from its community members. As of today, early adopters have already contributed many valuable tools and reference datasets to the platform, and there are many more in the works! Here is a preview of what you can find on precisionFDA today:
NA12878 benchmark calls made by NIST (Genome in a Bottle v2.19)
NA12878 benchmark calls made by Illumina (Platinum Genome v8.0.1 and an updated kmer-filtered v7.1.0)
HuRef (J. Craig Venter) benchmark calls made by Roche/Bina
NA12878 exome test calls made by the Broad Institute
NA12878 whole-genome sequencing and test calls made by the Garvan Institute (using the Illumina HiSeq X Ten)
Software assets and apps for simulation and evaluation using VarSim, added by Roche
An app for local ancestry analysis with RFMIX, added by Stanford
Additional early members of the precisionFDA community:
Baylor College of Medicine
Emory Genetics Laboratory
Human Longevity Institute
Above everything else, precisionFDA is a community, where people can collaborate, communicate, and even argue for the future of precision medicine. We are privileged to have been selected as the contractor for this pilot, and look forward to our collaboration with the FDA as the platform and community evolves. At this time, the precisionFDA platform includes features such as App Forking, Item Tracking, and Notes, which ignite collaboration, content expansion, and workflow validation and reproduction.
The Notes section, in particular, lets participants write and publish rich notes describing their thoughts and their work; for example, they can discuss how they used files, comparisons, and apps—which they can also attach to the note—to prove a certain point or to document a procedure. Community members can read what others have reported and access their attachments to take a closer look at that work or even reproduce it on their own.
We believe this new level of collaboration and reporting, together with everything else that precisionFDA has to offer, will define new frontiers for people to showcase to the FDA and to the rest of the community how to address the challenges of precision medicine in the 21st century.
We are pleased to announce that CareDx has selected the DNAnexus cloud genomics platform to support its clinical informatics for ongoing R&D activities, clinical trials, and registries, as well as the release of its analytically validated cell-free DNA (cfDNA) assay from their CLIA laboratory later this year.
CareDx, a molecular diagnostics company focused on the discovery, development and delivery of actionable diagnostic surveillance solutions for organ transplant recipients, will be expanding its footprint in the organ transplant space by delivering non-invasive tests based on the use of cfDNA as a biomarker for transplant surveillance. By combining the latest advances in genomics and bioinformatics with clinical research efforts, CareDx is able to generate high-quality data and evidence across the three major stages of diagnostic test development: analytical validation, clinical validation, and clinical utility. CareDx is one of the few CLIA laboratories that has satisfied the rigorous review of the FDA when it received clearance for its laboratory developed AlloMap™, a blood-based test used to monitor heart transplant recipients for the risk of cellular rejection. The DNAnexus and CareDx teams are working closely to complete the CLIA laboratory computational analysis pipeline for the CareDx cfDNA test.
CareDx’s cfDNA technology works by using next-generation sequencing (NGS) to measure donor-derived cfDNA as a percentage of total cfDNA. Earlier this year at the International Society of Heart and Lung Transplantation (Nice, France) and American Transplant Congress (Philadelphia, PA, US), CareDx reported levels of donor cfDNA rise in patient blood samples prior to and during heart and kidney transplant rejection and decrease following immunosuppressive treatment. You can read the respective abstracts here and here.
CareDx needed a way to rapidly implement a production ready solution to accelerate its cfDNA program. DNAnexus filled that role. DNAnexus scientists worked with CareDx to seamlessly integrate their proprietary NGS cfDNA pipeline to the DNAnexus cloud-based bioinformatics platform. Expediting cfDNA computational analysis efforts will enable CareDx to rapidly advance and scale this novel surveillance management solution to the transplant setting. The DNAnexus HIPAA compliant bioinformatics platform, with full versioning and auditing capability, supports CareDx’s diverse company activities, including R&D, its cell-free DNA Outcomes AlloMap Registry or D-OAR D-for heart transplants, and its DART clinical trial, a multi-center kidney transplant study using cell-free DNA. The DNAnexus platform provided significant time and cost advantages relative to building an on-premise solution.
CareDx’s cfDNA technology, based on pioneering studies by Drs. Stephen Quake and Kiran Khush and their colleagues at Stanford, could potentially be used for all solid organ transplants and provide additional benefits in simultaneous detection of infectious agents and the cancers associated with long term immunosuppression. We are excited to collaborate with CareDx as they expand upon their industry-leading organ transplant surveillance solutions and embark upon additional clinical opportunities and diagnostics markets.