WZeroRPC Logo (W0RPC)

Overview

WZeroRPC is a Web-based workflow orchestration and execution environment that allows multiple distributed workflows to be connected together using the Python programming language. Workflows can be executed synchronously or asynchronously and WZeroRPC provides mechanisms for interacting with data created within the workflows to support dynamic data-driven scenarios, where one workflow may process data until a certain condition occurs, and then another workflow maybe used to process the output data of the first workflow. The logic of how one coordinates the workflows is specified through a simple python API, which provides a powerful means of interacting with multiple coexisting synchronous and asynchronous workflow instances and connecting logical dependencies between them.

WZeroRPC supports the execution of directed acyclic graph (DAG) based workflows, where one controller node coordinates the distribution of workflow tasks to a collection of workers, and respects the dependencies between them. The DAG workflows are specified using the Python NetworkX package, which use a special WZeroRPC node object to provide an interface from a NetworkX graph nodes and the underling execution model. Dependencies are control-based dependencies that are topologically sorted and executed in that order to respect the flow between the nodes. Data dependencies between nodes can also be specified by using special tags that allow outputs of one node (elements of the return tuple from an RPC call) to the input of another (as an RPC argument). A similar tagging mechanism can be used to interface with data from a DAG as a whole. Multiple workflow instances are managed by a workflow manager, which uses a special plugin to inject customized python code for manipulating workflow instances in the way the programmer chooses. In this way, a programmer can create workflows, obtain workflow IDs and interface with them whilst reusing the WZeroRPC infrastructure for execution and visualization of the workflows.

Visualization is achieved through the use of Javascript Timeouts (polling) on the GUI to the Web backend, which in turn, requests the Workflow Manager to provide the status of all of its workflows. The Workflow Manager queries all of the workflow instances and each workflow and returns a graph to the Web front end containing its state, using the GraphViz textual format with embedded HTML and CSS layout information. The Web front end then uses Dagre and D3 to visualize the real-time progress of the workflow. The layout of the front end uses Semantic UI to render the GUI.

WZeroRPC Web Interface

More information

Installation

You will need to install the following packages:

And for the web server:

Some of the demos will also require the Matplotlib package.

Download

The WZeroRPC source code is available from: https://downloads.pf.itd.nrl.navy.mil/wzerorpc/

Contact Info

For further information on WZeroRPC, please contact wzerorpc_info@nrl.navy.mil.