Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bugfix_value_type_of_dataschema #986

Merged
merged 2 commits into from
Dec 16, 2023

Conversation

timmyb32r
Copy link
Contributor

it's improved copy of #985 - I made typo and couldn't fix previous branch

@duglin
Copy link
Contributor

duglin commented Nov 22, 2023

From the issue:

I occurred this problem when our users made correct cloudevents message with some non-golang sdk, and send it to our code.

can you elaborate on this? It appears that even with your change the URI will be serialized as a string, right? So why did things break when talking with a non-golang sdk?

@timmyb32r
Copy link
Contributor Author

timmyb32r commented Nov 29, 2023

It appears that even with your change the URI will be serialized as a string, right?

no, with my change DataSchema becames 'URI'. there are next code in project cloudevents/sdk-go:

func attributeFor(v interface{}) (*pb.CloudEventAttributeValue, error) {
    vv, err := types.Validate(v)
    //...
    switch vt := vv.(type) {
    case string:
        attr.Attr = &pb.CloudEventAttributeValue_CeString{
            CeString: vt,
        }
    case types.URI:
        attr.Attr = &pb.CloudEventAttributeValue_CeUri{
            CeUri: vt.String(),
        }
    //...
    return attr, nil
}

and types.Validate:

func Validate(v interface{}) (interface{}, error) {
    switch v := v.(type) {
    case bool, int32, string, []byte:
        return v, nil // Already a CloudEvents type, no validation needed.
    case *url.URL:
        if v == nil {
            break
        }
        return URI{URL: *v}, nil

so, in trunk now next code:
container.Attributes[dataschema], _ = attributeFor(e.DataSchema())
DataSchema() returns a string, and then string goes to attributeFor, and Validate returns string, and then

    case string:
        attr.Attr = &pb.CloudEventAttributeValue_CeString{
            CeString: vt,
        }

so, trunk creates type CeString

in my PR this becames:

dataSchemaStr := e.DataSchema()
uri, err := url.Parse(dataSchemaStr) // https://pkg.go.dev/net/url#Parse - returns *URL
if err != nil {
    return nil, fmt.Errorf("failed to url.Parse %s: %s", dataSchemaStr, err)
}
container.Attributes[dataschema], _ = attributeFor(uri) // passes *URL, which then in func 'Validate' becames type URI ne

so, *URL goes to 'attributeFor', and types.Validate make URI from *URL, and then in 'attributeFor' executes this branch:

    case types.URI:
        attr.Attr = &pb.CloudEventAttributeValue_CeUri{
            CeUri: vt.String(),
        }

This way DataSchema in trunk now CeString, and becomes CeUri. And documentations says 'DataSchema' must be URI, not string: https://github.com/cloudevents/spec/blob/main/cloudevents/spec.md#dataschema


also I'm fixing FromProto
and change

vs, _ := v.(string)
e.SetDataSchema(vs)

to

vs, _ := v.(types.URI)
e.SetDataSchema(vs.String())

so, FromProto in my PR withs with DataSchema==URI instead of string, what accordings to documentation: https://github.com/cloudevents/spec/blob/main/cloudevents/spec.md#dataschema

@timmyb32r
Copy link
Contributor Author

timmyb32r commented Nov 29, 2023

So why did things break when talking with a non-golang sdk?

my colleagues built message where DataSchema is URI, and cloudevents/sdk-go couldn't decode it, bcs it failed here: https://github.com/cloudevents/sdk-go/blob/main/binding/format/protobuf/v2/protobuf.go#L254

according to spec here must be URI, and cast to string makes an error - and this error ignores here via underlining, and empty string assigns to DataSchema field. I've got situation, when I have correct cloudevents message (according specification), and go-sdk corrupt the data, substitute empty value insteam of field value.

@duglin

@timmyb32r
Copy link
Contributor Author

up

@duglin

@duglin
Copy link
Contributor

duglin commented Dec 6, 2023

Sorry for the delay - and thanks for the explanation.
/LGTM

duglin
duglin previously approved these changes Dec 6, 2023
@duglin
Copy link
Contributor

duglin commented Dec 6, 2023

ping @embano1 and @lionelvillard for another set of eyes

@timmyb32r
Copy link
Contributor Author

ping @embano1 and @lionelvillard

Copy link
Member

@embano1 embano1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some minor things/question

dataSchemaStr := e.DataSchema()
uri, err := url.Parse(dataSchemaStr)
if err != nil {
return nil, fmt.Errorf("failed to url.Parse %s: %s", dataSchemaStr, err)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: use error wrapping

Suggested change
return nil, fmt.Errorf("failed to url.Parse %s: %s", dataSchemaStr, err)
return nil, fmt.Errorf("failed to url.Parse %s: %w", dataSchemaStr, err)

@@ -251,8 +256,8 @@ func FromProto(container *pb.CloudEvent) (*event.Event, error) {
vs, _ := v.(string)
e.SetDataContentType(vs)
case dataschema:
vs, _ := v.(string)
e.SetDataSchema(vs)
vs, _ := v.(types.URI)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: assuming this is guaranteed to be of type types.URI here due to your URI check earlier (no panic)?

Copy link
Contributor Author

@timmyb32r timmyb32r Dec 12, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good question. I've checked:
vs, _ := v.(types.URI) will return default constructed types.URI{}, and further called mechod .String() will return an empty string - so, it won't panic

@@ -109,6 +109,7 @@ func TestFromProto(t *testing.T) {
Type: "some.type",
Attributes: map[string]*pb.CloudEventAttributeValue{
"datacontenttype": {Attr: &pb.CloudEventAttributeValue_CeString{CeString: "application/json"}},
"dataschema": {Attr: &pb.CloudEventAttributeValue_CeUri{CeUri: "link"}},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can we use a proper URI for this test?

Copy link
Member

@embano1 embano1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@embano1
Copy link
Member

embano1 commented Dec 12, 2023

Can you please rebase onto main and also sign your commits or whatever is causing the DCO to fail?

@timmyb32r timmyb32r force-pushed the bugfix_value_type_of_dataschema2 branch from 99b13e7 to c54a03b Compare December 12, 2023 16:35
Signed-off-by: Tim Brunko <timmyb32r@gmail.com>
Signed-off-by: Tim Brunko <timmyb32r@gmail.com>
@timmyb32r timmyb32r force-pushed the bugfix_value_type_of_dataschema2 branch from c54a03b to 41e80f7 Compare December 12, 2023 16:46
@timmyb32r
Copy link
Contributor Author

@duglin @embano1 what we should also do to merge it? Idk why these last checks (Build/CloudEvents/Unit Test) wont pass :(

@embano1 embano1 enabled auto-merge December 16, 2023 08:34
@embano1 embano1 merged commit e620cf7 into cloudevents:main Dec 16, 2023
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants