Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Pub/Sub #154

Open
grayside opened this issue Jan 31, 2023 · 1 comment
Open

Add support for Pub/Sub #154

grayside opened this issue Jan 31, 2023 · 1 comment
Labels
api: eventarc Issues related to the googleapis/google-cloudevents-go API. priority: p3 Desirable enhancement or fix. May not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@grayside
Copy link
Contributor

Current Behavior

Pub/Sub is suppressed from the code generation process in generate-code.sh because Pub/Sub messages are incompatible with protojson parsing. This is because Pub/Sub messages include duplicate fields message_id/messageId, and publish_time/publishTime, which the protobuf spec indicates is invalid.

Expected Behavior

All Event Sources have type libraries.

Approach

Add a custom function that can be used to pre-process Pub/Sub payloads before unmarshalling.

We already have a function in the repository that can facilitate this that was put in place to enable testing:

func PreparePubSubMessagePublishedData(b []byte) ([]byte, error) {
var j map[string]interface{}
if err := json.Unmarshal(b, &j); err != nil {
return nil, fmt.Errorf("json.Unmarshal: %w", err)
}
m := j["message"].(map[string]interface{})
// Remove reserved @type field.
delete(m, "@type")
// Remove message_id field name conflict with messageId.
delete(m, "message_id")
// Remove publish_time field name conflict with publishTime.
delete(m, "publish_time")
j["message"] = m
return json.Marshal(j)
}

  • Move this function to the public API, preferably into the pubsubdata package. This will require a copying/templating process.
  • Unblock Pub/Sub from code generation
  • Update the code generator to use this function instead of the internal/testheper package

case params.TypePrefix == "google.events.cloud.pubsub.v1" && dataType == "MessagePublishedData":
params.PrepareFunction[dataType] = "testhelper.PreparePubSubMessagePublishedData"

@grayside grayside added type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. priority: p3 Desirable enhancement or fix. May not be included in next release. labels Jan 31, 2023
@product-auto-label product-auto-label bot added the api: eventarc Issues related to the googleapis/google-cloudevents-go API. label Jan 31, 2023
@ejcx
Copy link

ejcx commented Mar 8, 2023

@grayside I noticed this issue open but couldn't figure out how people are meant to parse pub/sub if not with this library? Is there a different library that should be used?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: eventarc Issues related to the googleapis/google-cloudevents-go API. priority: p3 Desirable enhancement or fix. May not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

No branches or pull requests

2 participants