Computational genomics workflows are regularly used to not only rapidly accelerate R&D in the field of genomics, but they are also increasingly used to make clinical diagnoses tailored to individual genomes. As DNAnexus has grown to support a large network of industries and collaborators, we have noticed that these workflows are often developed and shared across users and organizations on a global scale.
DNAnexus workflows currently provide the core functionality of allowing users to create and execute a computational workflow within a DNAnexus project. However, for users and organizations collaborating on multiple private or public projects, these local workflows may be less suitable for long-term maintenance in the context of larger organizations and consortia. In recognition of the need to manage workflows with a truly global network of collaborators, we are excited to introduce an additional suite of features that can be applied to objects we call ‘global workflows’.
As with DNAnexus applications, global workflows are published to a global space accessible by authorized users across projects. Like a Github repository or Docker repository, global workflows are versioned and updated with a globally unique name. Global workflows can be tagged, associated with broad categories (e.g. ‘read mapping’, ‘germline variant calling’, ‘somatic variant calling’, ‘tumor-normal variant calling’), and defined to run across cloud regions and providers. They can also be developed by a specified set of users and subsequently published or released to a larger set of authorized users who can run but not modify the workflow. Together, these features empower workflow developers to better share and advertise their workflows to a broad set of users and organizations across multiple regions and clouds.
A user creates a global workflow in essentially the same way as a regular workflow (see this tutorial for more details on how to create a global workflow). In fact, existing workflows on the DNAnexus platform can be converted to global workflows in a straightforward way. Since workflows written in CWL or WDL can be directly converted into workflows on our platform, these workflows can also be easily converted to global workflows. As a result, portable workflows can also be imported to our platform and used in a way that meets organizational needs for access control and collaboration at scale.
To illustrate the use of global workflows, we have published a public workflow available to all users of our platform. For example, from our CLI, you can run:
$ dx find globalworkflows
GATK4 Germline Best Practice FASTQ to VCF (hs38DH) (gatk4_germline_bp_fq_hs38dh),
v1.0.0
Here, you can see that there is a GATK4 best practice pipeline available for you to use. You can treat this workflow name like any other global applications on the platform. Examples for how to use these features can be seen in more detail here.
Workflow release management features were built by the Developer Experience team at DNAnexus. Thanks to the DNAnexus Science team for contributions to the design of this feature. Please see our documentation for a tutorial on how to use these features and contact support@dnanexus.com if you have any feedback or questions.
