Analyze implications and steps for webhooks support

[paoloantinori]

[nicolaferraro]

@kcbabo, can we consider #168 to be the user story for webhooks?

Part of it has been already addressed in the "set data shape" story. I'm collecting examples of webhook providers to ensure we're not missing something.

---

@kcbabo	2017-06-19	Epic, Web API

As a citizen user, I would like the ability to trigger an integration through an external HTTP request to a well-known URL. When creating an integration, I would select WebHook as the start connection and specify whether data is accepted via the WebHook. If data is accepted, I should have the option of defining the type of that data via JSON or XML schema.

Integrations triggered via WebHook must have a publicly available URL endpoint which can be discoverable via the iPaaS UI so that this information can be passed along to clients invoking the WebHook.

[nicolaferraro]

I've done some research on webhooks implementations (see other issues associated to the epic). I think we can distinguish a "basic" feature, that is outlined by the story above, and other "advanced" features that we may decide to add in the following iterations (e.g. handshake, validation, subscription).

For the "basic" functionality, I have the following questions (for @kcbabo):

• Should the URL be displayed in the integration detail page only? We should start @syndesisio/uxd work for this.
• Should it be displayed as soon as the integration is published/saved or the URL can be displayed when the integration is deployed (technically: "should we guess the domain used by Openshift for the route" vs. "we can read the host used by Openshift for the route")?
• We can create webhooks URL containing a token in the path (e.g. "httpz://mysyndesishost/webhook/A68gidjZAe7ejhfGsowp439djG28w6f"). This token can be a property of the endpoint that is autogenerated at the time of creating the integration and can be edited by the user (seems flexible and also easy to implement). Is it ok?
• I think we can consider the part about incoming data shape as done, since syndesis asks if the user wants to set the datashape when it's not defined (as for webhooks)
• Should the webhook "return" data to the caller? Usually the response of a webhook is not important, except for handshakes (Support for webhook challenge handshake #2666).

@kcbabo can you comment on the questions above?

[dongniwang]

@nicolaferraro - Cloud you provide some more information on what kind of deliverables you're expecting from the uxd? Is #168 the user story? Any specific use cases we're hoping to support? Hoping to get some more information so we can better scope the ux work. Thank you!

@paoloantinori - What is the timeline for this story? It would be great to understand this so we can prioritize the story.

cc: @amysueg @sjcox-rh

[nicolaferraro]

@dongniwang this story targets 7.1 (think it's "optional-but-really-nice-to-have" according to @paoloantinori plans).
I've pinged @kcbabo to provide more details on the story since it's a bit outdated.

From my point of view, the webhook connector should look like other connectors from a uxd side, except for the following points (we can discuss about them):

• In the integration detail page we should show the "URL" / "Public URL" property of the integration that is a url like "httpz://mysyndesishost/webhook/A68gidjZAe7ejhfGsowp439djG28w6f". It should be clearly visible but better not making it clickable since a click will trigger the integration. Integrations starting from a webhook will have such property (others may use this property too in the future).
• In the integration definition part there's a property that should be auto-generated. Since I think this is the first time such a property appears, we may think to just pre-populate it (as in the image) or to provide a different visual effect (I'm thinking to auto-generated properties on Openshift templates). This requirement should be confirmed by @kcbabo first.

[kcbabo]

@paoloantinori @nicolaferraro : where do you want me to file the user story for this? The original epic is closed and IIRC you don't want me to use Epic label any more for my user stories. I think it would be good to have a new user story that everyone can follow. I'll answer the questions on this issue, but still want to have a separate issue with the user story and that issue should be linked to any UX and eng activity.

[kcbabo]

@nicolaferraro Answers below ...

Should the URL be displayed in the integration detail page only? We should start @syndesisio/uxd work for this.

The full webhook URL will be unique per integration, so it needs to be displayed per integration. I think the best place to display that would be in the integration editor over the webhook start endpoint. We may also want to consider annotating the description of the integration in the integration listing to include the webhook URL.

Should it be displayed as soon as the integration is published/saved or the URL can be displayed when the integration is deployed (technically: "should we guess the domain used by OpenShift for the route" vs. "we can read the host used by Openshift for the route")?

I was thinking we would only have one route exposed that services all webhook endpoints. So that route will be something like https://webhook-app-proj11111.6a63.fuse-ignite.openshiftapps.com and every integration that starts with a webhook would have a unique context path, e.g.
https://webhook-app-proj11111.6a63.fuse-ignite.openshiftapps.com/app1
https://webhook-app-proj11111.6a63.fuse-ignite.openshiftapps.com/app2

We can create webhooks URL containing a token in the path (e.g. "httpz://mysyndesishost/webhook/A68gidjZAe7ejhfGsowp439djG28w6f"). This token can be a property of the endpoint that is autogenerated at the time of creating the integration and can be edited by the user (seems flexible and also easy to implement). Is it ok?

Yeah that works, but editable by the user could present problems if the name they choose clashes with another webhook endpoint. Even if we check for name collisions in the current environment, users could get bit by this when they export integrations into other environments.

I think we can consider the part about incoming data shape as done, since syndesis asks if the user wants to set the datashape when it's not defined (as for webhooks)
Should the webhook "return" data to the caller? Usually the response of a webhook is not important, except for handshakes (#2666).

I used the term webhook pretty loosely in my user story. :-) What we're really talking about here is a dump HTTP listener. Users should be able to feed it data and get data back. Agree that the type issue has already been addressed by shapeless connector support in Syndesis.

[kcbabo]

I vote for auto-generate path for now. Avoids a lot of potential problems. We can always add user-defined paths later.

One additional detail here is how we plan to handle securing the webhook endpoint. Will users be able to build that into the route configuration in OpenShift (same auth for all webhooks) or do we need to add it directly to Syndesis?

[kcbabo]

@gaughan - FYI

[nicolaferraro]

I close this as story and UX design have their own tickets now.