proto: store extension values according to protobuf data model #746
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cybrcodr
approved these changes
Nov 14, 2018
| if err := proto.SetExtension(cloneTestMessage, pb.E_Ext_Text, proto.String("hello")); err != nil { | ||
| panic("SetExtension: " + err.Error()) | ||
| } | ||
| if err := proto.SetExtension(cloneTestMessage, pb.E_Greeting, []string{"one", "two"}); err != nil { |
There was a problem hiding this comment.
Do you also need to verify that clone makes a deep copy on this?
neild
reviewed
Nov 14, 2018
The current API represents scalar extension fields as *T and repeated extension fields as []T. However, this is not an accurate reflection of the protobuf data model. For scalars, pointers are usually used to represent nullability. However, in the case of extension scalars, there is no need to do so since presence information is captured by checking whether the field is in the extension map. For this reason, presence on extension scalars is not determined by checking whether the returned pointer is nil, but whether HasExtension reports true. For repeated fields, using []T means that the returned value is only a partially mutable value. You can swap out elements, but you cannot change the length of the original field value. On the other hand, the reflective API provides methods on repeated field values that permit operations that do change the length. Thus, using *[]T is a closer match to the protobuf data model. This CL changes the implementation of extension fields to always store T for scalars, and *[]T for repeated fields. However, for backwards compatibility, the API continues to provide *T and []T. In theory, this could break anyone that relies on memory aliasing for *T. However, this is unlikely since: * use of extensions themselves are relatively rare * if extensions are used, it is recommended practice to use a message as the field extension and not a scalar * relying on memory aliasing is generally not a good idiom to follow. The expected pattern is to call SetExtension to make it explicit that a mutation is happening on the message. * analysis within Google demonstrates that no one is relying on this behavior.
neild
approved these changes
Nov 15, 2018
dsnet
added a commit
that referenced
this pull request
Nov 15, 2018
This hack has now been upstreamed in the v1 codebase; there is no longer any need to maintain the code in v2. See #746 Change-Id: I2c59d11303f5465e65190d893c422c088c647691 Reviewed-on: https://go-review.googlesource.com/c/149659 Reviewed-by: Herbie Ong <herbie@google.com>
dsnet
added a commit
that referenced
this pull request
Nov 27, 2018
The current API represents scalar extension fields as *T and repeated extension fields as []T. However, this is not an accurate reflection of the protobuf data model. For scalars, pointers are usually used to represent nullability. However, in the case of extension scalars, there is no need to do so since presence information is captured by checking whether the field is in the extension map. For this reason, presence on extension scalars is not determined by checking whether the returned pointer is nil, but whether HasExtension reports true. For repeated fields, using []T means that the returned value is only a partially mutable value. You can swap out elements, but you cannot change the length of the original field value. On the other hand, the reflective API provides methods on repeated field values that permit operations that do change the length. Thus, using *[]T is a closer match to the protobuf data model. This CL changes the implementation of extension fields to always store T for scalars, and *[]T for repeated fields. However, for backwards compatibility, the API continues to provide *T and []T. In theory, this could break anyone that relies on memory aliasing for *T. However, this is unlikely since: * use of extensions themselves are relatively rare * if extensions are used, it is recommended practice to use a message as the field extension and not a scalar * relying on memory aliasing is generally not a good idiom to follow. The expected pattern is to call SetExtension to make it explicit that a mutation is happening on the message. * analysis within Google demonstrates that no one is relying on this behavior. Change-Id: Ieec4c3ba8176aabe7c3139d12741c7fcc46ba83d Cherry-Pick: github.com/golang/protobuf@52132540909e117f2b98b0694383dc0ab1e1deca Original-Author: Joe Tsai <joetsai@digital-static.net> Reviewed-on: https://go-review.googlesource.com/c/151436 Reviewed-by: Damien Neil <dneil@google.com>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The current API represents scalar extension fields as *T and
repeated extension fields as []T.
However, this is not an accurate reflection of the protobuf data model.
For scalars, pointers are usually used to represent nullability. However,
in the case of extension scalars, there is no need to do so since presence
information is captured by checking whether the field is in the extension map.
For this reason, presence on extension scalars is not determined by checking
whether the returned pointer is nil, but whether HasExtension reports true.
For repeated fields, using []T means that the returned value is only a partially
mutable value. You can swap out elements, but you cannot change the length of
the original field value. On the other hand, the reflective API provides methods
on repeated field values that permit operations that do change the length.
Thus, using *[]T is a closer match to the protobuf data model.
This CL changes the implementation of extension fields to always store T
for scalars, and *[]T for repeated fields. However, for backwards compatibility,
the API continues to provide *T and []T. In theory, this could break anyone
that relies on memory aliasing for *T. However, this is unlikely since:
field extension and not a scalar
The expected pattern is to call SetExtension to make it explicit that a mutation
is happening on the message.