Use buf for proto generation

[tomhjp]

The documented protobuf generation steps no longer work. We could make some smaller incremental updates, but I thought it was a good opportunity to modernise the tooling and migrate to buf. However, I'm happy to close this PR and do the smaller changes if people aren't happy that this is safe enough.

I've split the PR into 3 commits to make the review easier:

1. Hand-written changes to switch over to buf.
2. Machine-generated output of 3 buf generate commands.
3. Fixup of a few build errors due to the new requirement to embed unimplemented servers in each implementation.

Additional notes/oddities:

• I ran through all of the gRPC examples and checked they still worked manually. The only exception is examples/grpc/plugin-go-netrpc which didn't work before and still doesn't. It's easy to get working with a small code change, but I left it alone for this PR as it's somewhat off topic.
• I've split the buf files into 3 packages because 2 of the examples had lots of type conflicts and I wanted to avoid a larger refactor that fixing this would have needed.
• I couldn't get buf to generate the python pb into the same location it was before, however the new location seemed like a logical improvement so I didn't fight it too hard.
• The python plugin's instructions were missing a few steps so I shored those up a bit and added a requirements.txt to pin its dependencies

[johanbrandhorst]

This looks great, well done!

[johanbrandhorst]

So easy 😍. If you want, we could use remote plugins the same way for the go and go-grpc plugins and drop those tools from this repo. Usually the argument against it is that we don't want to depend on the BSR but since we're doing this for python anyway, why not?

[tomhjp]

Yeah I think that would be a nice win to make it easy to use consistent plugin versions too. Added in e340950.

[johanbrandhorst]

These are somewhat unusual. I might suggest using the "managed mode" feature of Buf's generation instead: https://buf.build/docs/generate/managed-mode. That would mean you can drop this option in all the files and buf would automatically set it to something sensible. It'll almost certainly look something like this:

managed:
  enabled: true
  go_package_prefix:
    default: github.com/hashicorp/go-plugin/examples/bidirectional/proto # go.mod path + proto output path

[tomhjp]

Is there a way to do this but keep the slightly odd grpctest package name the same? I know it's only a test package, but it is technically part of the public API so I'd like to be quite strict about avoiding breaking changes, just to make the safety of the change easy to evaluate.

[swenson]

Looks good to me in general. I want to do some testing to see if plugins built against this still work as expected against ones built against previous versions of go-plugin.

[swenson]

LGTM

I pulled this down and did some testing that this still worked with Vault (all tests seem to pass for me locally, and plugins built using the old go-plugin seem to work with a newer Vault).

[tomhjp]

Thanks!