Do we want a new optional "mustunderstand" core attribute?

[duglin]

Extensions such as #1317 are much more useful if there's some guarantee that consumers will honor them.

We could introduce a new attribute of "mustunderstand" that lists the extensions that a consumer must understand in order to process the message further.
I don't know how this would be played out in SDKs, and it could be tricky to add within v1. But let's discuss.

From @jskeet / xregistry/spec#194

[jskeet]

Oops - apologies for filing this in the wrong repo!

[deissnerk]

Isn't it hard to determine which attributes a consumer must understand? What exactly would that mean? Attributes like partitionkey or sequence might not be interesting for all consumers who then can safely ignore them.
I see the point for #1317 , but there won't be a guarantee that consumers will honor the extension. It would be a cooperative concept like "Please ignore the event, if you don't know what attribute xyz means".
Alternatively, we could also think about marking this in message or endpoint definitions in xRegistry.

[duglin]

12/5 call

• what's the impact on the SDKs? Should have a common way of expressing it
• Ben has some usecases to think about - will add them

[xibz]

Two immediate use cases where I dont know how this would work:

• logging
• proxy services that rely on cloud events but whatever being passed to does understand attribute

I think the logging one is less complicated and could probably be fine following mustunderstand. However, the proxy one is a little interesting cause it will now require knowledge of what other systems support, I would think at least.

[duglin]

Couple of things that I wonder about (mainly brainstorming ramblings):

• are there any existing REST APIs that have the mustUnderstand concept? I'm pretty sure there are some that ban unknown extensions, but that's not the same thing. I can't think of one, so I'm wondering if that's a sign that this might not be that big of an issue
• Kind of related to the previous one, if we look at something like HTTP headers (e.g. "Accept"), some APIs (e.g. github) require this header to be there sometimes and will reject the request if it's not present. This sounds like a candidate for the mU semantics, yet there are a few things I think about:
  • they seem to manage just fine today w/o mU
  • the server decides what is required not the client. This might be a critical aspect of this.
  • in many ways, CE is just a transport level spec. If understanding of the mU attributes is critical to processing the event correctly, shouldn't that info be encoded as part of the event itself? We often talk about how CE attributes are there to "help" processing, and not necessarily replace data within the event itself. Which is why we've also talked about how (often) CE data is extracted from the event payload itself so it can be found more easily.
  • Having said that, I do see potential value is putting common info (like data classification) outside of the event (like ancillary metadata) so each event can focus on the more specific business logic - but then that means CE data needs to be kept with the event and in binary mode that's kind of annoying. Does this raise the question of whether we should have a "data classification" extension and instead suggest that it be encoded as part of the event itself?
• events are typically one-way messages, or more aptly named "fire-n-forget" messages. This means that any error generated by the processing of the event will likely not be seen by the sender. In which case the best that could be hoped for is for the sending dev to notice some log message in the receiver - which can happen today w/o this mU concept, if that's something they want people to take notice of.
• does the mU concept violate the notion of "extensions" being "extra optional stuff" ? Meaning, if the attribute is that important then perhaps a new CE-like spec should be defined with that attribute being mandatory?

While the above sound negative, I'm not necessarily against the idea, but I do think we need consider the above questions... in particular, do we really need something that everyone seems to live fine w/o today?

I'm not inclined to bump CE's major version number over this, so my proposal is that we tag this as a potential "v2.0" issue and consider it when we have enough other changes to warrant a v2.0 spec. At worst, if we add it to v2.0 I think it would be an optional feature that senders might choose to never use... so no harm done in adding it, and (I suspect) a minor coding effort for receivers.

[duglin]

12/12 call

• Mark suggested that perhaps we make it an extension today (under "v1.0"), to see if we can get some feedback before we consider making it a 'core' attribute for v2.0

Defer to Jan for more discussions