Support routing to different eventlisteners based on request content

[bobcatfish]

Expected Behavior

In our recent working group meeting, @mnuttall described his use case where he wants to have triggering for many repos, where each needs to have different eventlistener configuration (). He would like a way to minimize the number of ingresses and running interceptors that are required to support this, e.g. something like:

• One endpoint for all GitHub events to hit
• One interceptor invoked to make sure that the request is valid
• A way to route traffic to different eventlisteners based on request content (e.g. if the github repo is X and it is a pullrequest, use eventlistener Foo, but if it's a cron - not a PR - use eventlistener Bar)

Actual Behavior

In order to support this today, you can:

• Have many eventlisteners, each with an interceptor that has to both 1) check if the request is valid and then 2) check if the request meets the criteria for that eventlistener. All events must go to all eventlisteners
• ?

Additional Info

Question for @mnuttall: do you want to have many eventlisteners configured running in this scenario, or would you rather have one eventlisteners with many bindings, and a way of mapping from the request to the binding? (And of course is there anything I missed / got wrong about your use case? :D)

[mnuttall]

Hi @bobcatfish thank you for writing this up.

One EventListener makes event routing simple - it gets all events. However it's then routing to multiple triggers across multiple repositories, and so maintaining it - via GitOps or otherwise - can become contentious. We may start with one uber-EventListener but I think we need to evolve beyond it.

Ideally I'd like the owners of a Pipeline for a specific repository to also own its Trigger artifacts - TriggerTemplate, TriggerBinding, EventListener. That would leave them more in control of a simple GitOps process to manage their EventListener - but now we have multiple EventListeners.

If we assume that all inbound webhooks arrive at a single collector, then that collector can do one of two things:

• spray all inbound (webhook) events to all listeners, each of which must then filter out the bulk of them, or
• select a particular event listener, perhaps by labels on its associated Service, to route the payload to the right target - which could then skip some validation.

So we have to have smarts either in an EventListener - 'is this for me?' - or in front of a set of EventListeners. I would like to be able to implement the 'spray' pattern, which makes EventListeners simple 'subscribers to an event feed.' So I'm left with wanting a simple condition or similar syntax on an EventListener - that makes it clear, from the EventListener's definition, what it's filtering for. A slightly less pleasant alternative is to allow multiple validators, and then have one take the job of 'check that target (repository) matches a given field in the event payload.' It's less pleasant because it's less declarative - less obvious from simply looking at the object - I need to understand the function and API of the validator in question.

[vtereso]

One endpoint for all GitHub events to hit | A way to route traffic to different eventlisteners

I suppose it would be possible to send a request to an "interceptor" that is actually an EventListener? A CEL expression (or some custom interceptor) could read the request and mutate/decorate the data to construct the necessary EventListener service name. However, I believe we will need to introduce support for interpolation within interceptors (or by some such method) so we could pass to the generated service name. If that is sound, I think it would be possible to have a single exposed endpoint that routes internally.

I think there is some discussion around what an appropriate response is for interceptors, which might tie into auditing of such a design.

Alternatively, handling any number of event types can already be handled with multiple triggers.

One interceptor invoked to make sure that the request is valid

Whether through use of the core interceptors (occurring within the EventListener sink pod) or as a separate interceptor, this should already be possible.

[dibyom]

/kind feature

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[tekton-robot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Send feedback to tektoncd/plumbing.

[tekton-robot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.

/lifecycle rotten

Send feedback to tektoncd/plumbing.

[tekton-robot]

@tekton-robot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Send feedback to tektoncd/plumbing.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[bobcatfish]

/remove-lifecycle rotten

@dibyom do you think this makes sense to include in the roadmap?

[bobcatfish]

Oh it already IS in the roadmap https://github.com/tektoncd/triggers/blob/master/roadmap.md

/lifecycle frozen

[vdemeester]

/area roadmap