Skip to content

Commit

Permalink
proto: Add a descriptor subpackage.
Browse files Browse the repository at this point in the history
This provides a more reasonable API for obtaining a FileDescriptorProto and
DescriptorProto for a given proto.Message — a process that is currently possible
(but undocumented) using the public proto API.

The major use case for obtaining a DescriptorProto is to decode custom message
options, which are encoded as extensions on the MessageOptions submessage.
(See https://developers.google.com/protocol-buffers/docs/proto#customoptions.)

Fixes #179.
PiperOrigin-RevId: 139356528
  • Loading branch information
bcmills committed Nov 17, 2016
1 parent 62e782f commit cf10ca0
Show file tree
Hide file tree
Showing 2 changed files with 125 additions and 0 deletions.
93 changes: 93 additions & 0 deletions descriptor/descriptor.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
// Go support for Protocol Buffers - Google's data interchange format
//
// Copyright 2016 The Go Authors. All rights reserved.
// https://github.com/golang/protobuf
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

// Package descriptor provides functions for obtaining protocol buffer
// descriptors for generated Go types.
//
// These functions cannot go in package proto because they depend on the
// generated protobuf descriptor messages, which themselves depend on proto.
package descriptor

import (
"bytes"
"compress/gzip"
"fmt"
"io/ioutil"

"github.com/golang/protobuf/proto"
"google.golang.org/genproto/protobuf"

This comment has been minimized.

Copy link
@awalterschulze

awalterschulze Dec 10, 2016

Are we introducing dependencies into golang/protobuf now?

This comment has been minimized.

Copy link
@bcmills

bcmills Dec 12, 2016

Contributor

There isn't really an alternative for this subpackage. It needs access to the generated protocol message, and (because of the registration mechanism in the proto package) generated messages must only appear in one (canonical) location. See #178 for more detail.

Note that this dependency only affects the descriptor subpackage, not proto itself.

This comment has been minimized.

Copy link
@awalterschulze

awalterschulze Dec 13, 2016

Wouldn't the alternative be to have the descriptor live in golang/protobuf?
I have my descriptor.proto and descriptor.pb.go living inside gogo/protobuf. Its been working really well for years.

This comment has been minimized.

Copy link
@bcmills

bcmills Dec 13, 2016

Contributor

There is, in fact, already a copy of descriptor.pb.go in the golang/protobuf repo (here).

The problem is that that isn't the canonical location for the protobuf types: if the user of this package needs to refer to the protobuf message types anywhere else in their program, they'll be importing the canonical protobuf package and the redefinition will cause a failure during registration.

This comment has been minimized.

Copy link
@awalterschulze

awalterschulze Dec 13, 2016

So why are we Registering in the first place?
This just seems to break alot of people's setups and we never needed it before.

This comment has been minimized.

Copy link
@bcmills

bcmills Dec 13, 2016

Contributor

So why are we Registering in the first place?

proto.MessageType and proto.MessageName require it. So does extension support, and ptypes.DynamicAny, and there may be other cases I'm not thinking of at the moment.

This just seems to break a lot of people's setups

Let's be concrete. How is this breaking your (or anyone else's) setup?

and we never needed it before.

That's true of any change that addresses a feature request. This one was for #179.

This comment has been minimized.

Copy link
@awalterschulze

awalterschulze Dec 17, 2016

Ok this is starting to become quite long, so I'll start a proper issue.
#268

)

// extractFile extracts a FileDescriptorProto from a gzip'd buffer.
func extractFile(gz []byte) (*protobuf.FileDescriptorProto, error) {
r, err := gzip.NewReader(bytes.NewReader(gz))
if err != nil {
return nil, fmt.Errorf("failed to open gzip reader: %v", err)
}
defer r.Close()

b, err := ioutil.ReadAll(r)
if err != nil {
return nil, fmt.Errorf("failed to uncompress descriptor: %v", err)
}

fd := new(protobuf.FileDescriptorProto)
if err := proto.Unmarshal(b, fd); err != nil {
return nil, fmt.Errorf("malformed FileDescriptorProto: %v", err)
}

return fd, nil
}

// Message is a proto.Message with a method to return its descriptor.
//
// Message types generated by the protocol compiler always satisfy
// the Message interface.
type Message interface {
proto.Message
Descriptor() ([]byte, []int)
}

// ForMessage returns a FileDescriptorProto and a DescriptorProto from within it
// describing the given message.
func ForMessage(msg Message) (fd *protobuf.FileDescriptorProto, md *protobuf.DescriptorProto) {
gz, path := msg.Descriptor()
fd, err := extractFile(gz)
if err != nil {
panic(fmt.Sprintf("invalid FileDescriptorProto for %T: %v", msg, err))
}

md = fd.MessageType[path[0]]
for _, i := range path[1:] {
md = md.NestedType[i]
}
return fd, md
}
32 changes: 32 additions & 0 deletions descriptor/descriptor_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
package descriptor_test

import (
"fmt"
"testing"

"github.com/golang/protobuf/descriptor"
tpb "github.com/golang/protobuf/proto/testdata"
"google.golang.org/genproto/protobuf"
)

func TestMessage(t *testing.T) {
var msg *protobuf.DescriptorProto
fd, md := descriptor.ForMessage(msg)
if pkg, want := fd.GetPackage(), "google.protobuf"; pkg != want {
t.Errorf("descriptor.ForMessage(%T).GetPackage() = %q; want %q", msg, pkg, want)
}
if name, want := md.GetName(), "DescriptorProto"; name != want {
t.Fatalf("descriptor.ForMessage(%T).GetName() = %q; want %q", msg, name, want)
}
}

func Example_Options() {
var msg *tpb.MyMessageSet
_, md := descriptor.ForMessage(msg)
if md.GetOptions().GetMessageSetWireFormat() {
fmt.Printf("%v uses option message_set_wire_format.\n", md.GetName())
}

// Output:
// MyMessageSet uses option message_set_wire_format.
}

0 comments on commit cf10ca0

Please sign in to comment.