Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Canonical marshalling for imported messages #34

Open
fdymylja opened this issue Nov 3, 2021 · 1 comment
Open

Canonical marshalling for imported messages #34

fdymylja opened this issue Nov 3, 2021 · 1 comment
Milestone

Comments

@fdymylja
Copy link
Contributor

fdymylja commented Nov 3, 2021

Our deterministic marshalling implementation enforces canonical marshalling for messages.
This is achieved because we generated that code with pulsar codegen.

But what happens when we import a message that was not generated with cosmos-proto?

Ex:

message GeneratedWithCosmosProto {
  google.Protobuf.FileDescriptor not_generated_with_cosmos_proto = 1;
  string field = 1;
  map<string, int64> map_field = 2;
  ...
}

How do we detect that the import is not generated with pulsar from marshal codegen, and then how do we marshal it in a canonical way without impacting performance much?

Use cases:

  • ORM will most likely save objects protobuf file descriptors into state using the google.Protobuf.FileDescriptor message.
  • Timestamp and Duration proto

NOTE:
We can enforce deterministic marshalling for imported objects, protobuf marshal options supports this, but there is no guarantee that that deterministic marshalling follows https://github.com/cosmos/cosmos-sdk/blob/master/docs/architecture/adr-027-deterministic-protobuf-serialization.md

@aaronc
Copy link
Member

aaronc commented Dec 15, 2021

So my proposed approach is to create a canonical marshaler in the cosmos-proto runtime that generated code will call on imported messages. Since the official marshaler is reflection based too, there should be a way to keep performance similar.

To simplify this we can probably use the https://pkg.go.dev/google.golang.org/protobuf/reflect/protorange package which also can be used for amino json and maybe unknown field rejection in the future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: 📋 Backlog
Development

No branches or pull requests

3 participants