-
Notifications
You must be signed in to change notification settings - Fork 267
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
coprocessor for query planning #4966
Comments
Hi @samuelAndalon. That looks like an interesting project and it's nice to see how the coprocessor API can be extended to provide functionality like this. This is a problem area that we have thought about several times over the last couple of years and we have started to form some opinions and work up some ideas about how to provide this kind of functionality. Here are some of the main problems we'd like to be able to address:
(there are more, but this paints a broad picture) If I test out your idea across those 6 areas it would probably look a little like this:
That's a fairly positive picture, it looks like this would help reduce latency and provide a way to scale QP resource independently. Could we do better? For a long time we've been aware of the benefits of caching query plans to be shared between routers. That provides a variety of benefits, but this is still a sub-optimal use of resource, since there is still a duplication of query planning across a fleet of routers. One approach could be to provide a "standalone query planner with associated cache". The standalone planner could be deployed as a resilient component with pluggable cache and accessible by a fleet of routers. How would this score against our criteria?:
This would require a careful evaluation of the implications of item 4 and 6, but, on the whole, I'm confident that a dedicated, distributed, caching query planner would be a much more scaleable solution than any of the alternatives: status quo or sidecar planning. One other drawback would be the fact that it would potentially be a SPOF, but careful design of the distributed planner should be able to minimise that risk. For this kind of scenario: scalability is probably far more important than straight ahead performance, but we would continue to support query planning in the router for those deployments where performance far outweighs scalability requirements. |
The advantage of having a query planning service is that it can be shipped as a sidecar, or as a completely independent service out of a "pod", if is deployed as a single entity then multiple router instances will be calling the same "query planner service"
If query planner service is separately deployed, then, this wont be a problem.
Based on some benchmarks that i performed, Bun is extremely fast in comparison with node and deno. (check issue where i shared some screenshots), Query Planning is 100% CPU, synchronous execution, and the Bun runtime is extremely fast, yes it is still javascript, but similar to the experimental parallelism feature in the router, Bun provides workers that will help to execute query plans in parallel.
I believe this can be tuned, HTTP2, TCP Keep Alive connections, etc. on top of that we have the cache in routers that will avoid making same requests over and over again. The best thing about this is that based on he needs the router user will decide how to ship the router and query planner service,
both options are agnostic of the router, and router will just need a query planner url, for the planning coprocessor. |
Is your feature request related to a problem? Please describe.
Coproccesors allows hooking into the router lifeclycle allowing extending the behavior of the app, router service, supergraph service, execution service, subgraph service.
Describe the solution you'd like
It would be nice if query planning could be decorated with a coprocessor that way we could decide where and how a query plan will be built.
If a coprocessor for query planning is not detected then spawn the deno process, and if it is, don't spawn it.
wrote an small Bun server capable of building query plans and ran some benchmarks comparing it against node.
node on top, bun bottom, 100tps during 10s.
unlike other the coprocessors, query planning coprocessor wont be called as much as cache for query plans will be in the main router app
Describe alternatives you've considered
rely on federation, however, building query plans and updating supergraphs is synchronous, it doesnt soon like a bad idea do leverage those responsibilities to a "sidecar".
Additional context
Add any other context or screenshots about the feature request here.
The text was updated successfully, but these errors were encountered: