Revisit PR 218 to see if partitionkey extension is still needed

[duglin]

See: #218

Once the Kafka transport PR ( #337 ) is resolved revisit the decision to add the partitionkey extension - it might not actually be needed, and if not the group should consider removing it per the discussion in #218.

[bluemonk3y]

I have seen cases where the partitionKey information is not part of the payload and cases where it is. I can also see a case where a partitionKey might be augmented with a partitionKeyExtractor function; to rewrite the partitionKey or even replace it to allow for data-stream collocation or data distribution purposes.

Pros and Cons when removing the partitionKey.

Pros:

1. Potentially pushes down transport concerns during Cloud Event generation where key extraction will work without payload modification
2. Reduces the number of Cloud Event spec attributes

Cons:

1. Potentially force the producer to be aware of transport concerns (i.e. when embedding the partition key data)
2. Potential overhead and secure-key access where a secured data payload is concerned

[duglin]

@hschaffner ^^^

[clemensv]

Can we close this?

[duglin]

based on what @hschaffner said on the previous call, yes I think we can. Heinz let me know if I'm remembering incorrectly.

[hschaffner]

Clemens: As I mentioned a while back you can close this. My concern was that adding the optional field for a partition key was not really applicable to the CE specification. With Kafka the key is part of the payload since Kafka records are name/value pairs. The optional key is part of the payload, but like the Kafka Topic it is defined at transport binding. It really is not really part of the CE Spec for the object definition just like Topic, Exchange or Queue name for other transports. However, if you want to add it as an optional header, I have no issue, however, it is unnecessary. This is part of my general concern of adopting things into the spec that are not generally applicable. Another case in point is the Avro serialization as an alternative to the structured JSON payload. Avro is predominantly used for Hadoop V2 and because of the early relationship it has been adopted by Kafka. But do you need another structured payload and serialization defined? Especially when it is so limited in focus? Even more interesting by the fact the Kafka library already supports a bi-directional Avro/JSON conversion? Heinz From: Clemens Vasters <notifications@github.com> Reply-To: cloudevents/spec <reply@reply.github.com> Date: Tuesday, July 2, 2019 at 5:13 AM To: cloudevents/spec <spec@noreply.github.com> Cc: Heinz Schaffner <Heinz.Schaffner@solace.com>, Mention <mention@noreply.github.com> Subject: Re: [cloudevents/spec] Revisit PR 218 to see if partitionkey extension is still needed (#430) Can we close this? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#430?email_source=notifications&email_token=ACM4KYJZ4BRWHQI5PJP62M3P5ML2NA5CNFSM4HL7IG62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZAT2OA#issuecomment-507591992>, or mute the thread<https://github.com/notifications/unsubscribe-auth/ACM4KYNT5632ZVJRBQBNV7TP5ML2NANCNFSM4HL7IG6Q>.