Support a pipeline of execution

[afrittoli]

Expected Behavior

As a triggers user I would like to be able to run a series of common operations on the event payload before splitting into the various triggers.

Examples are:

• (1) checking the git provider token
• add extra content / perform pre-processing of the payload to add fields required for CEL filters downstream. Examples:
  • (2) extract commands from a github comment
  • (3) append the PR details to the payload when missing
  • (4) verify if the PR / comment submitter is authorized to run CI checks

Actual Behavior

In the CI setup I have a single event listener with multiple triggers:

• one for pull_request events on any repo (for shared CI jobs)
• one for comment events on any repo (for shared CI jobs)
• one for pull_request events on repo X (one for each tektoncd repo)
• one for comment events on repo X (one for each tektoncd repo)

Each of these need (1) and (4). All the comment triggers need (2) and (3) too.
Today the interceptor have to be executed again and again for each trigger for each event.
(3) and (4) are implemented via custom interceptors. (2) is done in the pipelines downstream.

Alternatives

a. Implement some way in triggers to run common filters / overlays before getting into the triggers.
b. Use a single trigger, and run all the checks on it. From there, trigger a task / pipeline that receives the whole payload, enhances it as required, and forwards the whole event to the event listener again. This first task could then do extra things like setting up source caches for downstream tasks.
c. Leave things as they are and implement a caching layer in the interceptors so that we do not have to go all way back to github all the time

/kind feature

[wlynch]

Thanks for the feature request!

Initial reaction - this seems like a good goal (in particular, for webhook validation). We're looking to keep the EventListener largely agnostic, deferring to Interceptors to handle user customization and other things like credentials and caching. Having an EventListener level hook for user defined interceptors might be one way to approach this. We've been discussing adding catalog support for interceptors to make using Interceptors asier (e.g. you could use a predefined interceptor for comment -> PR resolution instead of writing your own), which EventListenters could presumably also use.

There's a few things we need to work through. In particular, we'll need to figure how how this impacts debugging for triggers (particularly w/ multi-tenant EventListeners where users have access to Triggers, but not necessarily to the EventListener).

As a short term solution, alternative (c) seems most reasonable to me. (b) has the downside of decoupling events, which can make it harder to debug/trace what's happening. (a) is this FR, which we'll continue to look into, but isn't available at the moment.

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[dibyom]

Ref #945, #940

[afrittoli]

This seems partly related too: #205 - the feature talks about chaining event listeners - but the idea to have an initial common filtering logic is the same.

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[tekton-robot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle rotten

Send feedback to tektoncd/plumbing.

[tekton-robot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen with a justification.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/close

Send feedback to tektoncd/plumbing.

[tekton-robot]

@tekton-robot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen with a justification.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/close

Send feedback to tektoncd/plumbing.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.