Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Record the decision on Broker and Trigger #862

Merged
merged 6 commits into from
Mar 14, 2019
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
166 changes: 166 additions & 0 deletions docs/decisions/broker-trigger.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
# Broker and Trigger model (for 0.5.0)

Decision date: 6 March 2019
Copy link

@duglin duglin Mar 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add a list of terms/components with a one-liner for each one - e.g.:

  • Event Producer - creates events based on occurrences. E.g. github
  • Event Consumer - destination for events from an Event Producer. E.g. the application processing an event. Typically, this is the final "sink" in the processing flow of an Event.
  • Event Source - a helper entity that can be used to connect an Event Producer to the Kn eventing system and a destination (sink) for those events (e.g. an Event Consumer or a Broker)
  • Event Source Service - a Knative Service that is the endpoint to which Event Producers send events. This is created by the Event Source
  • Sink - a generic term to mean the destination for an event being sent from a component.
  • Broker - an entity that can be a Sink. Subscriptions can be created for the Broker to register interest in Events that flow through the Broker. Brokers are typically used to provide loose coupling between the incoming events and Event Consumers - e.g. it may buffer/queue events.
  • Trigger/Subscription - a registered interest in events that flow through a Broker. This will typically include a Sink (where to send the Events) and an optional filter to control which Events are sent. It may buffer/queue events.

Not sure about both the Broker and Trigger buffering things.... did we talk about that? I noticed below it talks about the Trigger doing the buffering.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think including this would help. In particular I like is that the terms have a concrete example associated with them.

I don't think 'Event Source Service' is needed. Or if it is included, it should not mandate that it is a Knative Service (e.g. it could be a K8s Deployment).

Event Producer should also specify that the event produced does not need to be a CloudEvent.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Conceptually, each Trigger has an independent buffer. The implementation of where the buffer lives in the data plane is a bit mushier, as I expect that the Broker and Trigger will collapse into the same data-plane objects in many cases.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added some updated definitions (for example, the "Event Source Service" is called a "Receive Adapter", and might be a Deployment or StatefulSet instead of a Knative Service, depending on the Event Source implementation).

I left out the following two definitions, because they are defined later in the document in more detail:

  • Broker: an event dispatch object which acts as an event Sink. Triggers can
    be created on the Broker to register interest in Events that are delivered to
    the Broker. Brokers are used to provide loose coupling and content-based
    routing between incoming events and Event Consumers - e.g. it may
    buffer/queue events.
  • Trigger a registered interest in events that flow through a Broker. This
    will typically include a Sink (where to send the Events) and an optional
    filter to control which Events are sent. It may buffer/queue events.


## Define High-Level Objects Related to Eventing

The goal of eventing is to provide an abstract "bucket of events" which event
consumers can sample from using filters on CloudEvent
[context attributes](https://github.com/cloudevents/spec/blob/master/spec.md#context-attributes).
Event producers can deliver events into the bucket without needing to understand
the exact mechanics of routing to specific consumers. Event producers and
consumers are decoupled temporally but also through a deliberately black-box
abstraction which can optimize the underlying routing layer dynamically.

## Definitions

- **Event Producer**: a system which creates events based on occurrences. E.g.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If an application (Event Consumer) produces an event in response to an event to process, do we need to mention it here? As in, only wondering if we should clarify that transformations are possible with the current system.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think transformations will muddle this more than clarify; today, we can treat this as an Event Consumer and an Event Producer.

GitHub (the hosted service).
- **Event Consumer**: a destination which receives events. E.g. the application
processing an event. Typically, this is the "sink" in the processing flow of
an Event, but could be a data processing system like Spark.
- **Sink**: a generic term to mean the destination for an event being sent from
a component. Synonym for Event Consumer.
- **Event Source**: a helper object that can be used to connect an Event
Producer to the Knative eventing system by routing events to an Event
Consumer. E.g. the
[GitHub Knative Source](https://github.com/knative/eventing-sources/tree/master/pkg/reconciler/githubsource)
- **Receive Adapter**: a data plane entity (Knative Service, Deployment, etc)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with this definition because it implies to be its more like a proxy. It "performs the event routing". And in that scenario the consumer would expect things like the CE-source to be something related to the original event producer. However, I know this view is not universally shared. So, we need to decide... is a Receiver Adapter just a piece of middleware or does it become the Event Producer ? And if the answer is "it depends on the Event Source" then let's call that out so it's clear that event consumer can't assume one model for all Receive Adapters.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're actually thinking of the type, not source context attribute?

Some guidance from CloudEvents would be helpful here; I think currently CloudEvents describes middleware (including routers) as "Event Consumers" and then "Event Producers". 😉

In all seriousness, I think this is something important to agree on, but I think it will also require (at least) a second decision process and document if Knative wants to make a recommendation for "Event Sources" as defined here. I think "Event Importer" would probably have been a better term. Fortunately, we haven't burned the "Importer" term yet...

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, I do mean "source" - assuming you're talking about this sentence:

And in that scenario the consumer would expect things like the CE-source to be something related to the original event producer.

See: cloudevents/spec#404 for my thoughts on this.

But, in general I view Knative's job w.r.t. event sources as utilities to help the user subscribe to event producers and to get those events delivered to their app. To the app, the fact that Kn is in the picture at all is irrelevant to the app doing its job of processing the event. And, in fact, asking the app to know about Kn makes it harder for them to move the code some place else. As I've mentioned in some slack chats, if the event producer already generated CloudEvents then these event adapters would be more like proxies. So, in that light, the fact that they need to do "some work" to create the CloudEvent isn't material to what's going on from an app's POV - its just necessary busywork to comply with CE. The adapter is not meant to really add any semantic value beyond compliance with CE - so therefore, making the CE appear like it came from the adapter instead of the real event producer/source goes against that view.

which performs the event routing from outside the Knative cluster to the
Knative eventing system. Receive Adapters are created by Event Sources to
perform the actual event routing.

## Objects

The following objects are the minimum set needed to experiment and collect
customer feedback on the Eventing object model, aka an MVP (including rough YAML
definitions; final naming and field definitions subject to change based on
implementation experience).

### Broker

A Broker represents a "bucket of events". It is a namespaced object, with
automation provided to provision a Broker named `default` in
appropriately-labelled namespaces. It is expected that most users will not need
to explicitly create a Broker and simply enabling eventing in the namespace to
create the `default` Broker will be sufficient. (The `default` broker will be
provisioned automatically by a controller upon appropriate namespace annotation.
This controller will _reconcile_ on the annotation, and so will recreate the
`default` Broker if it is deleted. Removing the annotation will be the correct
way to remove the reconciled Broker.) Assuming that the Broker model is
successful, future recommendations for Source CRD authors would be to default
the Source's `spec.sink` attribute to point to the `default` Broker in the
namespace.

Historical storage and replay of events is not part of the Broker design or MVP.

Broker implements the "Addressable" duck type to allow Source custom resources
to target the Broker. Broker DNS names should also be predictable (e.g.
`default-broker` for the `default` Broker), to enable whitebox event producers
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this implies that the "default" broker MUST always be present? e.g. if someone deletes it the system will recreate it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a k8s watch set up on the namespaces and if the appropriate label is found there, then the Broker is created. So if a user manually deletes the Broker it will be recreated.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't read this as requiring the default Broker to be present. Rather, that when a whitebox event producer is given a Broker to send events to, they can fully construct the DNS name for that Broker, without needing to get the Broker object itself.

From earlier in the doc:

It is expected that most users will not need to explicitly create a Broker and simply enabling eventing in the namespace to create the default Broker will be sufficient.

So we think that most of the time it will be the default Broker, but nothing requires it to be. Nor does anything require the default Broker to exist.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a mention earlier about the mechanism for provisioning the default broker, which is (as Ville suggests) that the broker is created by reconciling the namespaces with the appropriate annotation.

to target the Broker without needing to reconcile against the "Addressable" duck
type.

#### Broker Object definition

```golang
type Broker struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty"`
Spec BrokerSpec `json:"spec,omitempty"`
Status BrokerStatus `json:"status,omitempty"`
}

type BrokerSpec struct {
// Empty for now
}

type BrokerStatus struct {
Conditions duckv1alpha1.Conditions `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type"`

Address duckv1alpha1.Addressable `json:"address,omitempty"`
}
```

### Trigger

A Trigger represents a registration of interest in a filtered selection of
events delivered to a Broker which should be forwarded to an Addressable Object
or a URL. For the MVP, Triggers may only target Addressable objects in the same
namespace.

#### Buffering
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did we decide that buffering is done in the trigger instead of the broker? I can see an argument for either.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the difference between buffering in one vs the other?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Conceptually, the buffering applies to each Trigger -- if you have the following objects:

Broker B
Services X and Y
Trigger T1 matching type=foo pointing to X created at t1
Trigger T2 matching all events pointing to Y created at t4

And the following sequence of events delivered to the Broker:
t0: type=foo id=123
t2: type=bar id=124
t3: type=foo id=125
t5: type=foo id=126
t6: type=bar id=127

Then X will receive 125 and 126, and Y will receive 126 and 127.


Triggers are assumed to buffer (durably queue) filtered events from the Broker
evankanderson marked this conversation as resolved.
Show resolved Hide resolved
from the time the Trigger is created, and to deliver the buffered events to the
target Addressable as it is able to receive them. Delivery to the target
Addressable should be at least once, though backing storage implementations may
set limits on actual retries and durability.

#### Filtering

An event consumer which is interested in several sets of events may need to
create multiple Triggers, each with an ANDed set of filter conditions which
select the events. Trigger selection criteria are independent, not aggregated in
the Broker. In the case of multiple Triggers with the same target whose filter
conditions select an overlapping set of events, each Trigger which selects an
incoming event will result in a separate delivery of the event to the target. If
multiple delivery is undesirable, developers should take care to not create
overlapping filter conditions.

At some later point, we expect that a more complex set of filtering expressions
will be supported. For the initial MVP, all match or exact match on the
CloudEvents
[`type`](https://github.com/cloudevents/spec/blob/master/spec.md#type) and
[`source`](https://github.com/cloudevents/spec/blob/master/spec.md#source)
attributes will be supported.

#### Trigger Object Definition

```golang
type Trigger struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty"`
Spec TriggerSpec `json:"spec,omitempty"`
Status TriggerStatus `json:"status,omitempty"`
}

type TriggerSpec struct {
// Defaults to 'default'.
Broker string `json:"broker,omitempty"`
Filter *TriggerFilter `json:"filter,omitempty"`
// As from Subscription's `spec.subscriber`
Target *SubscriberSpec `json:"target,omitempty"`
}

type TriggerFilter struct {
// Exact match expressions; if empty, all values are matched.
Type string `json:"type,omitempty"`
Source string `json:"source,omitempty"`
}

type TriggerStatus struct {
Conditions duckv1alpha1.Conditions `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type"`
SubscriberURI string `json:"subscriberURI,omitempty"`
}
```

## Backing documents

[2019-02-22 Decision Making -- UX targets](https://docs.google.com/spreadsheets/d/16aOhfRnkaGcQIOR5kiumld-GmrgGBIm9fppvAXx3mgc/edit)

Accepted decisions (2+ votes)

- Have a "Broker" object
- Default sink on sources
- Enable with namespace/default Broker
- Have a "Trigger" object
- "filter" subfield (not top-level filters)
- "filter" has exact type match
- "filter" has exact source match
- target is same-namespace
- Registry requirements are not clear enough to include in this decision.
- Future user stories could change this and motivate a particular
implementation.

[UX Proposals Comparision](https://docs.google.com/document/d/1fRpM4u4mP2fGUBmScKQ9_e77rKz_7xh_Thwxp8QXhUA/edit#)
(includes background and discussion)