Level 3 RPC support

[zenhack]

Some discussion came up on the mailing list about adding Level 3 RPC support:

https://groups.google.com/forum/#!topic/capnproto/iXuK8wjmG8o

Louis was looking to arrange a video call around this; @zombiezen I know you don't have a ton of bandwidth for this project these days, but thought I'd ping to see if you'd be interested in joining us? No pressure.

I don't know Louis's GitHub handle, but I'll cross-link this issue on the mailing list.

[zombiezen]

If there's someone looking to get ramped up on contributing to the Go package, I'm happy to pass on knowledge, but otherwise I'm going to have to decline due to other commitments.

[lthibault]

I don't know Louis's GitHub handle, but I'll cross-link this issue on the mailing list.

@zenhack I'm here 🙂

If there's someone looking to get ramped up on contributing to the Go package, I'm happy to pass on knowledge

@zombiezen I'm quite interested in maintaining this package, and would definitely appreciate a guided tour. Happy to work around your schedule as much as possible.

[zombiezen]

Excellent! Left a reply on the mailing list so you've got my email address.

[taylorjdawson]

@lthibault have you assumed the role of package maintainer yet? I am also interested in helping 🙌
cc @zombiezen

[lthibault]

Hi Taylor, cc @zenhack and @RamakrishnaChilaka, who has recently expressed interest in maintaining this project as well. It seems like a small maintenance team is forming, which is great! The answer to your question is “yes”, but I haven’t yet had the opportunity to get much done. However, I have a bit of time and am quite eager to get the ball rolling on this, especially if I’m not going at it alone. In terms of how to proceed, I suggest we pick up where I left off in the Fall, namely: implementing level-1 embargos (which occur during path resolution, a.k.a. “path shortening"). It would be best to set up a conference call where I can share my notes, step through the code together, and split the work amongst ourselves. @zenhack — any chance you might be able to join us? Last time I looked at this I got hung up on a few implementation details, which I’m sure you’ll be able to clarify. In the meantime, I would urge you to brush up as much as possible on the following: 1. The RPC spec <https://github.com/capnproto/capnproto/blob/master/c++/src/capnp/rpc.capnp> 2. The stub implementation for senderPromise <https://github.com/capnproto/go-capnproto2/blob/v3/rpc/rpc.go#L1088> (N.B.: v3 branch) What time-zone is everybody in? Ian and I are US-Eastern. - Louis

…

On Feb 4, 2021, at 7:55 PM, Taylor Dawson ***@***.***> wrote: @lthibault <https://github.com/lthibault> have you assumed the role of package maintainer yet? I am also interested in helping 🙌 cc @zombiezen <https://github.com/zombiezen> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#160 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACIDYEJFSD5JZCWQ5NYQGTDS5M6ZTANCNFSM4RYY3XWQ>.

[RamakrishnaChilaka]

I am in IST, but I am comfortable with US Eastern time. Thanks. On Fri, 5 Feb 2021 at 9:11 PM, Louis Thibault <notifications@github.com> wrote:

…

Hi Taylor, cc @zenhack and @RamakrishnaChilaka, who has recently expressed interest in maintaining this project as well. It seems like a small maintenance team is forming, which is great! The answer to your questions I “yes”, but I haven’t yet had the opportunity to get much done. However, I have a bit of time and am quite eager to get the ball rolling on this, especially if I’m not going at it alone. In terms of how to proceed, I suggest we pick up where I left off in the Fall, namely: implementing level-1 embargos (which occur during path resolution, a.k.a. “path shortening"). It would be best to set up a conference call where I can share my notes, step through the code together, and split the work amongst ourselves. @zenhack — any chance you might be able to join us? Last time I looked at this I got hung up on a few implementation details, which I’m sure you’ll be able to clarify. In the meantime, I would urge you to brush up as much as possible on the following: 1. The RPC spec < https://github.com/capnproto/capnproto/blob/master/c++/src/capnp/rpc.capnp > 2. The stub implementation for senderPromise < https://github.com/capnproto/go-capnproto2/blob/v3/rpc/rpc.go#L1088> (N.B.: v3 branch) What time-zone is everybody in? Ian and I are US-Eastern. - Louis > On Feb 4, 2021, at 7:55 PM, Taylor Dawson ***@***.***> wrote: > > > @lthibault <https://github.com/lthibault> have you assumed the role of package maintainer yet? I am also interested in helping 🙌 > cc @zombiezen <https://github.com/zombiezen> > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub < #160 (comment)>, or unsubscribe < https://github.com/notifications/unsubscribe-auth/ACIDYEJFSD5JZCWQ5NYQGTDS5M6ZTANCNFSM4RYY3XWQ >. > — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#160 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALY3BJ4R3Y6X7EJMWMCEUWDS5QGUPANCNFSM4RYY3XWQ> .

-- CONFIDENTIALITY NOTICE: This E-Mail is intended only for the individual or entity to which it is addressed and may contain information that is confidential. If you have received this communication in error, do not distribute. Please delete the original message and immediately notify the sender.

[zenhack]

I'd definitely be interested in joining, let's use jitsi again. I have a preference for early afternoon EST, but am flexible.

[lthibault]

@zenhack Awesome! In the meantime, I'll add you to our Slack community, as I think some real-time coordination is helpful.

[taylorjdawson]

@lthibault Excellent! I am on the west coast, PST. Thanks!

[lthibault]

'Morning, @taylorjdawson !

You should have an invitation in the gmail account listed in your GitHub profile 🙂 . I'll re-send it, just in case.

[zenhack]

We're finally ready to break ground on this! There are some preliminary things we should think about, and some design questions. In particular:

• I've already addressed this, but recvCap: use the vine for thirdPartyHosted caps. #451 adds support for using vines, which we'll want to ease upgrades.
• We may want to consider finally addressing rpc.Conn doesn't support promise capabilities #2 first, and maybe add a corresponding API to generate non-answer promises. This would make path shortening much more useful.
• There are a few parts of the implementation I want to audit & think through the soundness of. In particular:
  • rpc.embargo blocks in some cases, which I think could lead to deadlocks, and I have a hunch might be more likely in a multi-party context.
  • We should stare at the way Client.resolveHook works in light of: https://github.com/capnproto/go-capnproto2/blob/f07bb8d6a744921cadc667ca5e4740e78de84228/std/capnp/rpc.capnp#L686-L709 In particular, I'm not sure resolveHook respects the "only one hop" shortening constraint; we need to think through how to make that work properly. This may also be a good time to think through some of the synchronization design in the main package.
• We'll want to define the main logic on top of an abstraction like the VatNetwork described in rpc.capnp, but we also will need to define at least one real implementation for it to be of any use.
  • We may want consider looping Kenton in on the design of that, to get buy-in for compatibility with any future implementation in C++ (and likely others).
  • Since BN is funding the work on this, I'd obviously like to hear from @lthibault what the environment they're using this in is like, and factor in those constraints.
  • I'd like to be able to use both websockets & webrtc at some point so we can have vats in the browser use 3PH. The fact that webrtc doesn't give vats an obvious stable identifier to work with introduces some challenges, and I'd like to think about how these might be solved before we commit to a design that prevents it.
  • It might be nice to design this in a way that different transports can be negotiated (with vats having preferences) within the same VatNetwork. Maybe using a public key as an identifier for the vat and having a signed statement or such that tells other vats how to connect to it.
  • I do want to think a bit about how to make interop with an ocapn bridge work smoothly as we design this. At the last pre-standardization meeting Jessica gave a presentation on how goblins currently does this, which is worth digesting & reviewing: https://share.tube/w/44DhG6pcEyCFgnUHhxnUTn

Probably some of the above should be split out into their own issues.