pb-rust: Serde via prost + pbjson

[jleightcap]

Summary

sigstore-rs requires the serialization of compiled strcutures/enums, currently via Serde.

• protoc_rust types do not implement Serialize/Deserialize. protoc_rust provides JSON support via internal reflection, however this approach is incompatible with existing sigstore-rs Serde derives, and proved incomplete with google.api (specifically google.api.field_behavior).
• prost is an alternative builder to compile from .proto. It has JSON serialization support via Serde.
• pb-json provides Serde derives for google.api.

Release Note

Documentation

[jleightcap]

Encoding HashOutput (uses only primitive types),

use sigstore_protobuf_specs::dev::sigstore::common::v1::HashOutput;

fn main() {
    let ho = HashOutput { algorithm: 0i32, digest: vec![1u8] };
    let serialized = serde_json::to_string(&ho).unwrap();
    println!("{}", serialized);
}

Serializes,

$ cargo run
{"algorithm":0,"digest":[1]}

[jleightcap]

Adding some tests. Done, similar to above but for the structures defined here.

[jleightcap]

Converting to draft, as Prost seems to have assumptions leading to miscellaneous serialization issues for protobuf.

[jleightcap]

Summarizing issues with protobuf compilation/serialization:

• compilation from protobuf source into rust structures is well-handled by both protoc_rust andprost.
• both have some level of support for serialization;
  • protoc_rust < 3.0 leverages serde by specifying #[derive(...)] per-compilation struct/enum.
  • Similarly with prost.
  • protoc_rust >= 3.0 (betas) move to an internal reflection library that seems to be more adept, but is mutually incompatible with serde and seems to be incomplete when used with the well-known types.
• a second approach of not manual specification of Serialize/Deserialize traits within this local crate
  • solves problems like field naming, different serializations of primitive types (u32 as number literal, u64 as string literal)
  • but is again incomplete when dealing with the well-known types and would require explicit dependencies on e.g. crono for Timestamp.
  • because this is external to the codegen, impl Serialize for, per defined protobuf structure, is (using SLOC as a metric) about equal in effort to manual definitions.

[jleightcap]

Superseded by #118