Figure out how to serve multiple services on the same socket address

[carllerche]

This will require some sort of service level router.

[vorner]

I was wondering if it could be done by implementing some kind of Either (or MultiEither with a vec instead of one with 2 options) with tower_h2::HttpService. Try the first one and if it returns NOT_IMPLEMENTED (or maybe add a „responsible_for“ method to find the correct one first), try the next one. The user could then compose them together.

Or did you have something more automatic in mind?

[carllerche]

I was wondering if it could be done with tuples and avoid a linear scan to find the service that matches.

I haven't really thought through it in detail yet.

[vorner]

Tuples ‒ well, I guess why not. It is probably more natural. Maybe a Vec or *Set too, in case one doesn't know the number at compile time.

Linear scan looked simpler. But the trait could be extended to some kind of fn prefix(&self) -> &str, the composing implementation could build a matcher out of them (aho-corasick, radix tree or something like that).

If that sounds in about the right direction, I could make a PR with a first shot (though I can't promise how soon).

[carllerche]

I would appreciate a PR to get things started! This is not on my short list of priorities.

[olix0r]

It seems likely that we could instrument tower-router (or something like it) to dispatch requests to the appropriate service base don the URI prefix

[carllerche]

The main issue w/ tower-router is that it requires all service instances to be of the same type. In this case, we need to be able to route to different service types.

One way to deal w/ this is to box the services, but this requires some runtime overhead. Another way would be to use a static routing strategy... this is trickier.

[illicitonion]

Has there been any new thinking here in the intervening 6 months? I'm looking to start using tower-grpc, and I could probably put something together if there's a direction in mind, but I'm not yet familiar enough with the general tower ecosystem to be designing solutions from scratch...

[carllerche]

Hey,

I have been doing a lot of exploration into how I would like to see "app development" go in the context of tower. This has mostly been done in the context of tower-web which is not yet public. If you are interested, we could talk some via Gitter and I can show you how I'm thinking it might apply back to tower-grpc.

I'm usually on http://gitter.im/tokio-rs/dev.

[thedodd]

Well, tower-web is public now 🙌 I’m gonna do some comparison and see what it would look like to borrow some of that logic and use it here.

[thedodd]

Per a lot of the discussion above:

1. Currently in tower-grpc, the generated tower::Service impl for the generated *Server has an fn call where the request's uri().path() is matched against method paths and then the appropriate rpc method is called based on the path.
2. This is exactly the type of "routing" like behavior that we will need, just on a higher level. So, we'll have to take this a few levels higher in the call stack.
3. When serve(socket) is called on the h2 Server instance, ultimately an instance of the user's gRPC service type is created via make_service and then its call method is executed. This takes us back to 1.. This may be the best location to hook into the system.

I'm thinking we may be able to introduce a new GrpcServiceRouter or the like.

• It could implement MakeService (I still need to study its generic constraints a bit more, but something along those lines).
• We could generate a new associated const, say SERVICE_PREFIX, for services in the generated grpc code so that we can effectively do routing.
• Users would create a new GrpcServiceRouter and add new Service instances to it. Perhaps this is where we could use the tuple pattern mentioned by @carllerche. Something like (&'static str, S) where the str is the service prefix const & S is MakeService<...> (still need to review this bit). Could store the tuples in a vector.
• When GrpcServiceRouter.call is called, it would evaluate the uri().path() of the request in the call method, just like a regular Service, but then create a new copy of the service to "route" to based on path prefix, and then call its call method.

DISCLAIMER: I still need to experiment with this, and the MakeService generic constraints, as they are now, may prove difficult for factoring in an abstraction like this ... we'll see.

Anyway, hopefully I will have time to experiment a bit with this. I am definitely interested in feedback on the pattern. Just let me know. Any and all feedback is welcome.

[jkryl]

@thedodd : your idea sounds good to me! Though I'm not a rust expert so take my opinion with a pinch of salt. Let me know if I can help with testing of an early prototype or perhaps help with coding in case you have your changes publicly available somewhere in a branch. thanks!

[jkryl]

We have been using a simple service router to run 2 services on a single endpoint for quite a long time already. It's rather simple but it is usable and IMHO worth sharing: https://github.com/jkryl/grpc-router . Have fun!

[thedodd]

@jkryl awesome! How likely do you think it would be to be able to factor such a pattern into upstream tower-grpc?

[LucioFranco]

@thedodd I believe this pattern actually belongs in something like the base tower repo, as you can abstract the recognizing part and be abstract over services. Though we are still not sure what the right approach is. If its to layer them like so or to use a macro to statically build out a service that can route.

[rlabrecque]

Just wanted to throw in my use case for this:

We have a pretty decent REST setup in go, and one thing that I'm missing with tower-grpc is routing. grpc-router above is working alright for my needs now, but I don't think it's going to scale to our REST setup.

The biggest part missing would be the ability to add routes later, and then get a list of the routes.

Here's what I'd want to write:

    let router_service = Router::new(
        GreeterService::new(),
        PingService::new(),
        SomeOtherService::new(),
    );

    // Add the APIListService and HealthCheckService after we created the router_service, and pass
    // it the router_service so that we can get a list of all of the services.
    let api_list_service = APIListService::new(&router_service);
    router_service.add(api_list_service);

    let healthcheck_service = HealthCheckService::new(&router_service);
    router_service.add(healthcheck_service);

    let mut server = Server::new(router_service);

    // Bind server to http2 server and continue as normally
    // ...

#[derive(Clone, Debug)]
pub struct APIListService {
    services: Vec<std::string::String>
}

impl APIListService {
    pub fn new(router: &Router) {
        server::APIListServer::new(APIListService { services: router.get_routes() } )
    }
}