OpenWhisk supports an open API, where any user can expose an event producer service as a feed in a package. This section describes architectural and implementation options for providing your own feed.
This material is intended for advanced OpenWhisk users who intend to publish their own feeds. Most OpenWhisk users can safely skip this section.
There are at least 3 architectural patterns for creating a feed: Hooks, Polling and Connections.
In the Hooks pattern, we set up a feed using a webhook facility exposed by another service. In this strategy, we configure a webhook on an external service to POST directly to a URL to fire a trigger. This is by far the easiest and most attractive option for implementing low-frequency feeds.
In the "Polling" pattern, we arrange for an OpenWhisk action to poll an endpoint periodically to fetch new data. This pattern is relatively easy to build, but the frequency of events will of course be limited by the polling interval.
In the "Connections" pattern, we stand up a separate service somewhere that maintains a persistent connection to a feed source. The connection based implementation might interact with a service endpoint via long polling, or to set up a push notification.
Feeds and triggers are closely related, but technically distinct concepts.
-
OpenWhisk processes events which flow into the system.
-
A trigger is technically a name for a class of events. Each event belongs to exactly one trigger; by analogy, a trigger resembles a topic in topic-based pub-sub systems. A rule T -> A means "whenever an event from trigger T arrives, invoke action A with the trigger payload.
-
A feed is a stream of events which all belong to some trigger T. A feed is controlled by a feed action which handles creating, deleting, pausing, and resuming the stream of events which comprise a feed. The feed action typically interacts with external services which produce the events, via a REST API that manages notifications.
The feed action is a normal OpenWhisk action, but it should accept the following parameters:
- lifecycleEvent: one of 'CREATE', 'READ', 'UPDATE', 'DELETE', 'PAUSE', or 'UNPAUSE'.
- triggerName: the fully-qualified name of the trigger which contains events produced from this feed.
- authKey: the Basic auth credentials of the OpenWhisk user who owns the trigger just mentioned.
The feed action can also accept any other parameters it needs to manage the feed. For example the cloudant changes feed action expects to receive parameters including 'dbname', 'username', etc.
When the user creates a trigger from the CLI with the --feed parameter, the system automatically invokes the feed action with the appropriate parameters.
For example,assume the user has created a mycloudant binding for the cloudant package with their username and password as bound parameters. When the user issues the following command from the CLI:
wsk trigger create T --feed mycloudant/changes -p dbName myTable
then under the covers the system will do something equivalent to:
wsk action invoke mycloudant/changes -p lifecycleEvent CREATE -p triggerName T -p authKey <userAuthKey> -p password <password value from mycloudant binding> -p username <username value from mycloudant binding> -p dbName mytype
The feed action named changes takes these parameters, and is expected to take whatever action is necessary to set up a stream of events from Cloudant, with the appropriate configuration, directed to the trigger T.
For the Cloudant changes feed, the action happens to talk directly to a cloudant trigger service we've implemented with a connection-based architecture. We'll discuss the other architectures below.
A similar feed action protocol occurs for wsk trigger delete, wsk trigger update and wsk trigger get.
It is easy to set up a feed via a hook if the event producer supports a webhook/callback facility.
With this method there is no need to stand up any persistent service outside of OpenWhisk. All feed management happens naturally though stateless OpenWhisk feed actions, which negotiate directly with a third part webhook API.
When invoked with CREATE, the feed action simply installs a webhook for some other service, asking the remote service to POST notifications to the appropriate fireTrigger URL in OpenWhisk.
The webhook should be directed to send notifications to a URL such as:
POST /namespaces/{namespace}/triggers/{triggerName}
The form with the POST request will be interpreted as a JSON document defining parameters on the trigger event. OpenWhisk rules pass these trigger parameters to any actions to fire as a result of the event.
It is possible to set up an OpenWhisk action to poll a feed source entirely within OpenWhisk, without the need to stand up any persistent connections or external service.
For feeds where a webhook is not available, but do not need high-volume or low latency response times, polling is an attractive option.
To set up a polling-based feed, the feed action takes the following steps when called for CREATE:
- The feed action sets up a periodic trigger (T) with the desired frequency, using the
whisk.system/alarmsfeed. - The feed developer creates a
pollMyServiceaction which simply polls the remote service and returns any new events. - The feed action sets up a rule T -> pollMyService.
This procedure implements a polling-based trigger entirely using OpenWhisk actions, without any need for a separate service.
The previous 2 architectural choices are simple and easy to implement. However, if you want a high-performance feed, there is no substitute for persistent connections and long-polling or similar techniques.
Since OpenWhisk actions must be short-running, an action cannot maintain a persistent connection to a third party . Instead, we must stand up a separate service (outside of OpenWhisk) that runs all the time. We call these provider services. A provider service can maintain connections to third party event sources that support long polling or other connection-based notifications.
The provider service should provide a REST API that allows the OpenWhisk feed action to control the feed. The provider service acts as a proxy between the event provider and OpenWhisk -- when it receives events from the third party, it sends them on to OpenWhisk by firing a trigger.
The Cloudant changes feed is the canonical example -- it stands up a cloudanttrigger service which mediates between Cloudant notifications over a persistent connection, and OpenWhisk triggers.
The alarm feed is implemented with a similar pattern.
The connection-based architecture is the highest performance option, but imposes more overhead on operations compared to the polling and hook architectures.