Adding a new field to grpc proto file breaks the functionality if client stub is not updated

[stefinie123]

Description:
When the grpc proto file is updated by adding a new field it breaks the existing client stubs which uses the the previous proto file definition.

Following error is observed:
Invalid protobuf byte sequence

Steps to reproduce:
Update the proto file by adding a new field to a type definition
Try to execute a client/service stub which was generated earlier

Affected Versions:
Swan lake alpha 1

OS, DB, other environment details and versions:

Related Issues (optional):

Suggested Labels (optional):

Suggested Assignees (optional):

[daneshk]

@stefinie123 can you explain the issue bit more, better if you can provide an example

1. can you share the change you did for the proto file definition.
2. did you regenerate the stub file on either in client or server-side.
3. did you change the generated stub code manually.

[stefinie123]

@daneshk Please find my answers below:

1. We did the following change in proto file where we added a new optional field is_anonymous to the request definition

    message ValidateUserRequest {
        string user_idp_id = 1 [(validate.rules).string.min_len = 1];
        string email = 2 [(validate.rules).string.email = true];
        string invitation_correlation_key = 3;
        bool is_anonymous = 4;
    }
   

2. The server side stub was regenerated but the client side stub was not regenerated
3. No

[daneshk]

@BuddhiWathsala can you check this

[BuddhiWathsala]

I was able to reproduce the issue. However, there is a design concern to discuss here. When there are unknown fields, what Java does is, Java creates an UnknownFieldSet and return it to the user. However, since we are only deserialising fields that are specifically mentioned in the proto file, we couldn't do such a thing in our implementation. In addition, since we represent proto messages using Ballerina closed record, returning unknown fields is against the design.

Therefore we should decide on a suitable approach:

• ignore the unknown fields if the user does not regenerate the proto file
• embed the unknown values to the record as anydata

@daneshk @stefinie123 WDYT?

[daneshk]

I was able to reproduce the issue. However, there is a design concern to discuss here. When there are unknown fields, what Java does is, Java creates an UnknownFieldSet and return it to the user. However, since we are only deserialising fields that are specifically mentioned in the proto file, we couldn't do such a thing in our implementation. In addition, since we represent proto messages using Ballerina closed record, returning unknown fields is against the design.

Therefore we should decide on a suitable approach:

• ignore the unknown fields if the user does not regenerate the proto file
• embed the unknown values to the record as anydata

@daneshk @stefinie123 WDYT?

IMO, It is ok to Ignore the unknown field as the recipient doesn't know those fields to retrieve the value.