What is RestFlow?

RestFlow is a dataflow programming language and runtime engine designed to make it easy for scientists to build and execute computational pipelines (see Key Features of RestFlow). These scientific workflows can include steps for automated sample handling, instrument control, data acquisition, and data analysis. Steps in a scientific workflow may invoke third-party scientific applications or be defined using any of a variety of scripting languages familiar to scientists including Python, Bash, Perl, and Tcl.

In addition to scheduling steps in a workflow and managing the flow of data through those steps, RestFlow organizes the intermediate and final data products of each run of a workflow in directory structures that make it easy to find and evaluate results.

RestFlow is named for its strategy for scientific software integration. It facilitates the integration of variable-based scripts and program units via the flow of data items whose values are at rest (see Dataflow Programming Concepts). Additionally, its use of URI templates to organize data produced during a workflow run is inspired by the REST paradigm.

RestFlow was originally developed by the Macromolecular Crystallography group at the Stanford Synchrotron Radiation Lightsource and the DAKS group at UC Davis as part of the AutoDrug project.