pogs: codegen

[zenhack]

As mentioned in #62, I'm working on table-based tests for go.sandstorm. For this to work I need pogs-style structs for basically every type in the sandstorm API. It goes without saying that doing this manually is rather tedious; it would be nice if go-capnpc could generate types usable with pogs.

[zombiezen]

Interesting. I suppose you could have an alternate code generator program that dumps out the structs you want, but I think it would be hard to have the transitive closure of types in general.

[zenhack]

Quoting Ross Light (2017-02-21 11:18:32)

I think it would be hard to have the transitive closure of types in general.

Not sure I see why?

[zombiezen]

Would you maintain a parallel tree of the imports for referenced pogs types, or would you inline all of them? Both have tradeoffs, but if you're using the tool in a semi-automated way, either could work.

[zenhack]

I had been thinking the former.

[zombiezen]

Either approach is fine. One thing to note: the way I use pogs will align closely (but not exactly) with an underlying schema, so some amount of manual intervention is to be expected. For an example, see https://github.com/zombiezen/mcm/blob/master/internal/catpogs/catpogs.go

[zenhack]

Just wanted to record a problem that's been in my head re: a problem that crops up with this and default arguments: suppose a schema adds a new field to a struct, which has a custom default argument. Updating the output of the code generator will silently change the value created by client code, as adding the field means we'd be setting the value to Go's zero value, rather than the default value which would have been effectively generated by the code that just didn't write the value.

[hspaay]

Below my 2c's from a newbie that is struggling with the steep learning curve. If it wasn't for the excellent help I received I would have thrown in the towel already and continued with the non-capability gRPC. That would have been a real shame 😄
In short, I think that the adoption of go-capnproto would be greatly helped by a higher level API that can work with plain-old-go-objects.

One example, currently you cannot easily use the generated types elsewhere in the code as they are tied to the RPC messaging. Few people would be willing to spread all the rpc messaging aspects in their code that is not doing any messaging. I certainly won't. The result is that the Go implementation of an app now has to add a duplicate type definition, one for plain Go and the other for the capnp-rpc. Ouch.

By far the most ideal solution is that the generated code can use plain old Go types. Maybe as a separate layer on top of the current low level API.

There might be a performance penalty but in most cases it might not be noticeable. Copying of fields would almost always happen anyways. Right now the burden is on the programmer. In most cases any performance degradation might not be noticeable with all the other processing going on in handling a request. In cases where performance is critical, the low level API can still be used by the experts.

I think this issue is underrated. Can this be made a top priority?

[zenhack]

I can't speak for others, but for myself: my own development priorities are primarily driven either by the needs of downstream projects I'm involved with, or by clients who are willing to pay for specific features. It's possible that someday this will end up at the top of my own list, but if you want to make this happen in a reasonable time frame, your best bets are either:

1. Get your hands dirty; I'm happy to to provide advice/feedback/review prs.
2. If you've got the budget for it, reach out about doing it on a contract basis.

[hspaay]

Fair enough. I don't have the budget for it, nor do I know of a possible sponsor.
As for option 1. That might be an option on the longer term. Right now my limited time is directed towards making my hiveot project work with capnp. Once it gets to a stable and usable setup, and my understanding of capnp has improved, I will consider digging in.