Feature Request: graphql Server Chaining #4
Comments
|
@JProgrammer This is a pretty interesting idea, but I wonder whether it could actually be easier to just have the user schema in the app server (exposing only the parts you want) and making the query from there. A graphQL server is essentially a proxy, and I'm not sure yet whether it makes sense to chain them, or whether it's better to just push it all into the gateway (the app server). Or maybe I misunderstood, and you're asking for a way to compose schemas rather than having one graphql server rely upon another for part of a query? |
|
@helfer The idea is that in the REST world you would have multiple small servers looking after small parts of the system, microservice, so that they can be independently upgraded, maintained, tested etc. So if graphql/apollo is replacing the data stack of REST then I would like to be able to make small graphql servers to take responsibility of the service logic/database access then in an application or aggregate server that sits across these servers to serve the individual client queries. |
|
I'm curious what the benefit would be of having the microservices use GraphQL to communicate rather than REST. In my current mental model, the microservices still speak REST, and there is a single GraphQL server that composes all of the data. But perhaps there are significant benefits to using GraphQL everywhere! |
|
I guess with graphql-tools just being a express middleware there is not a large overhead for microservices to offer both REST and GraphQL. With the end goal of apollo-server having meteor like reactivity having a server pass reactivity between servers will reduce load and distribute a single read/write operation across multiple clients and make a simple path for inter application communication. A lot like how meteor server can connect to another meteor server over DDP rather than having to share state over REST. |
|
Closing this, since it's almost three weeks old. Feel free to continue discussing it, and reopen if you have a specific use case that cannot currently be supported with the current implementation of apolloServer. |
|
I am looking at a use case that may be well served by such a feature. My interest is in open local government data (cities & counties) and, as an extension, in the ways that local government systems make data accessible internally as well. GraphQL strikes me as potentially a great way to provide such data both as a standard internally and via open data portals. Obviously the ability for clients to specify the shape of the data they want is important here (since the use cases that will employ the data aren't known in advance), but the validation and introspection capabilities of GraphQL are especially valuable. One of the interesting aspects of open government data is that cross-organizational (e.g., different departments or agencies) and even cross-jurisdictional (e.g., the county in which a city is located) data queries are important. In many cases this will be a one-off task and can be left to the client, but I expect some queries to be common and important enough that organizations should provide an aggregated view. This might be done by a city, combining multiple department sources into a common portal or even pointing, say, to county, state or federal sources of related data. Or it might be done by a 3rd party, say an investigative journalism organization or an advocacy group. It is certainly possible to create some kind of proxy server that could take care of this and perhaps that's the better way, but the fact that such a proxy would certainly need to be able to combine both GraphQL and REST endpoints makes me think that it would be better integrated directly into the GraphQL server. Also, optimizing such queries may be possible, but that again assumes built-in knowledge of the functioning of GraphQL, which resides best inside the same server. @JProgrammer - perhaps it's worth re-opening this issue to discuss such a use case? |
|
I'm also looking at a problem I believe would be served by this capability.
A large benefit I see is that I only need to express my schema once and in one format (gql), which is shared by my client, and the "end" servers that know how to service various sub-trees of the client's query. All my proxy server(s) should need to know is how to route subtrees of the query to org-specific services. |
The same benefits you get from an API gateway using GraphQL is true for an internal microservice like query batching or caching for example. Why would you go back to REST, just because it's not externally facing? |
|
I am interested in this as well. We have multiple subsystems each with a current REST Query API. Then there is a Gateway that has the public client REST Query API. The Gateway can do integration across the REST Query APIs. I think it would be great if the Gateway could be a GraphQL server that aggregated the schemas from each subsystem GraphQL server. Clients using the Gateway GraphQL endpoint could then query across all the subsystems. I guess what I am asking for is a datastore that can be another GraphQL server. I |
|
Coursera mentioned a similar approach in their graphql talk. I think this is THE missing piece in a micro-service architecture:
If anyone is interested to work on it, please hit me up :) |
|
This is something I might be interested in working on when we hit the need - at the moment we're building just the one API, but I expect we'll get there. As a side note, we're building some data infrastructure where the base-level GraphQL API is actually automatically generated from metadata - curious if anyone else is doing anything like this. Just did an article on Medium about our approach. |
|
I'm working on a small demo for this exact thing. I'll let you guys know if I have something to show. |
|
We have a micro-services architecture; we've built a graphQL endpoint for our apps to access (public) and then connect to a number of other services for varying things. We needed to build a new internal (private) service; we built this as another graphQL endpoint and used apollo-client on the server. This is working really well for us, I just love working with GraphQL over REST, and thanks to apollo-client there was no need to build a separate "client" application for a REST endpoint(s) The only thing that I found I was repeating my schema definitions (and all the descriptions) in both systems. I also made my internal graphQL quite "flat" with everything on the RootQuery or RootMutation and only my public facing GraphQL more graph like. |
|
The question for me is: Assuming we have a number of microservices with GraphQL APIs, how can we best implement a Gateway GraphQL API that offers queries across all the microservices? It seems that automatically merging GraphQL schemas isn't too difficult (if that is what you wish to do and excepting namespace issues) graphql/graphql-js#223 and we have a tool to help https://github.com/liamcurry/gql The missing piece, in my opinion, is how to delegate queries for subgraphs of the gateway API to the microservice GraphQL APIs. Can the appropriate resolvers just pass on their subgraph queries? Do we need a client (or clients) on the gateway? |
|
I'm going out on a limb here, and continuing on Ashley's thinking, but there would seem to be a need here to basically prune (or copy) parts of the graph for each microservice, right? So, to define the microservice, we'd need
I've also been contemplating a microservices architecture, not using GQL, but rather using gRPC (and automating this too), because gRPC has somewhat better (more official/ under one roof) support for different languages than GQL does and all the benefits GQL offers (but specifically for microservices). Plus, gRPC has type definitions which are language agnostic through Proto3. Well, it seems it does. 😄 I'm learning. This is all blue sky thinking on my part, but it shouldn't be too hard to build a parser, which translates GQL to proto3 (or the other way around!!!) and away we go! Scott |
|
@rogchap how did you implement apollo-client on the server? i'm interested in this too! i have a web server that i need to get connected to my data api via graphql instead of having to set up rest endpoints for server->server data calls. my current option is to hit the db directly from my web server to do some things and let the client hit the graphql server for everything else. it seems kind of silly to be hitting the db from 2 different places. |
|
It's very simple, we use graphql on an internal server and use apollo client on the other server. Although this does work, I'd recommend gRPC too. We're planning on refactoring to use gRPC for server-to-server and graphQL only as the public facing API |
|
One use case I thought about is to have one GQL server to handle auth & permission logic, chained with one handling the real data. So you authenticate with the public facing server, so that you can pass queries to private servers. I frown upon using gRPC, thus adding another library in my codebase. Using GQL only would be a benefit. Why can't GQL serve all our needs? |
It can, once all the necessary tooling is made available. 😄 The only question is, when will it come? Currently GraphQL doesn't support what gRPC does. When it does, then sure. It can cover all the API bases. Scott |
@servermeta While that was not one of the design goals of graqhql, I think there is not fundamental reasons it could not be a graph enabled rpc layer.
@smolinari What do you think is missing in Graphql compared tp gRPC? J |
|
@jnak - These points are off the top of my head and from my limited knowledge.
Scott |
|
@smolinari Interesting. With regard to 1. and 2., I think graphql took the route of building very idiomatic client and servers in each language by building a spec. While it is harder to bootstrap as an ecosystem, I think medium term it was right choice. When I look at https://github.com/chentsulin/awesome-graphql, I feel we are almost there. Regarding 3., I'm hopeful that graphql subscriptions (http://graphql.org/blog/subscriptions-in-graphql-and-relay/) can evolve into a solid bi-directional layer. Future looks bright :) |
|
@jnak - That's why I said "It can, once all the necessary tooling is made available." 😄 I love GraphQL's future too. Because, I believe it is built much more to be a client based API (if you can call it that). By that I mean, the client has much more flexibility in getting only the data it needs and defining the calls on its own. GraphQL is a great end-user(GUI client)/ server API system. This kind of API system isn't possible with gRPC or rather, gRPC is what I consider more server or service based and works better in a microservices environment. They say you can use gRPC for mobile clients, but I still think GraphQL beats it out in this area of API needs. Scott |
|
Here's a use case I think solving this issue would fulfill. I have a central graphql server that my client interacts with. Behind it are several legacy REST services. So far, so good. Next, we want to expose our legacy postgres database to our central server using something like postgraphql or join-monster. This is good because it can give a client access to arbitrary but safe queries. But here's the problem: we want to have our central server handle auth., and then pass the relevant graphql query along to the db afterwards. It would be nice if there were a way to grab that db subquery, I think. |
|
BTW we're calling this "Schema Stitching" and here's the implementation! ardatan/graphql-tools#382 hooray! please chime in there. |
|
@stubailo That's awesome!! Can't wait to see what people do with this! |
|
... and history is made |
…rver-env (#1259) * Export polyfills and types separately * More imports from apollo-server-env * Initial commit * Add .npmignore to avoid ignoring lib when publishing * 0.0.2 * Reorganize code and clean up GraphQLExtension interface * 0.0.3 * Add support for timing callbacks and add GraphQLExtensionStack * 0.0.4 * Downgrade target in tsconfig.json from es2015 to es5 * 0.0.5 * Bump `graphql` peerDependency. (#3) * 0.0.6 * Update dependencies * 0.0.7 * whenResultIsFinished fix for array results (#4) * 0.0.8 * [apollo-bot] Update the Issue/PR Templates with auto label (#6) * Bump `graphql` peerDependency. (#7) * Update `graphql` peer dependency range to allow 0.13.x. (#8) * Update `devDependencies` to latest versions. (#9) * dev: Update TypeScript to latest version, v2.7.2. * dev: Update `graphql` to latest version, v0.13.2. * dev: Update jest & dependencies to latest versions. * dev: Update type definitions for `graphql`, `node` and `jest`. * Allow `undefined` return values to `GraphQLExtension`'s `format()`. (#10) In some cases, it's conceivable that the `format()` method may need to abort its decision to provide extension information at runtime, in the event that it doesn't have the proper information to return a full-result. The `format` method already removed false-y results, so this simply changes the types to allow the same. * 0.0.9 * Fix lifecycle method invocations on extensions * 0.0.10 * Add changelog * Upgrade to TypeScript 2.8 Makes my editor integration happier (a bugfix in tsserver I think) * Add tslint and prettier Same configuration as apollo-engine-js * Remove magic from GraphQLExtensionStack constructor It's not hard to consistently pass in an actual extension object to this low-level API. * New extension API: didStart handlers return didEnd handlers This is a backwards-incompatible change: GraphQLExtension implementations and users of GraphQLExtensionStack (ie apollo-server-core) must change their implementations, if they implement any of the xDidStart/xDidEnd APIs. This allows "didEnd" handlers to refer to closure variables from the "didStart" handler rather than needing to store state on the extension. The new "didEnd" handlers run in the opposite order of the "didStart" handlers, so that they properly nest. * 0.1.0-beta.0 * Changelog * Add magic back into GraphQLExtensionStack constructor But now it actually gets more context (the execution arguments) and doesn't have to be a constructor. * 0.1.0-beta.1 * Export more types * 0.1.0-beta.2 * Fix lifecycle handlers to pass proper "this" * 0.1.0-beta.3 * Pass options directly to start handlers; eliminate factory again * 0.1.0-beta.4 * error handling in didEnd * 0.1.0-beta.5 * pass multiple errors to EndHandler * 0.1.0-beta.6 * add willSendResponse * 0.1.0-beta.7 * prettier * setFieldResolver for custom fieldResolver * reverse * get more initial options into requestDidStart * 0.1.0-beta.8 * 0.1.0-beta.9 * Actually, we already get the fieldResolver! * 0.1.0-beta.10 * work without extensionStack * 0.1.0-beta.11 * 0.1.0-beta.12 * Send errors to willResolveField callback * 0.1.0-beta.13 * willSendResponse can return a result * 0.1.0-beta.14 * Revert 1063be8..56912fc This reverts commit 1063be8..56912fc. * add PQ options to requestDidStart * 0.1.0-beta.14 * 0.1.0-beta.15 * Initialize an empty TypeScript/Jest package Template based on apollo-engine-js * Basic trace node structure building * basic timing * Checkpoint towards signature implementation The new signature implementation does not try to compress whitespace. * Basic signature implementation * progress towards actual reporting * basic checkpoint for reporting * 0.0.0-beta.1 * pull in @types/long, since it is in the external api * 0.0.0-beta.2 * get rid of Long * 0.0.0-beta.3 * debug log request what happened * 0.0.0-beta.4 * 0.0.0-beta.5 * correct url * 0.0.0-beta.6 * request headers * 0.0.0-beta.7 * leave out a few headers * 0.0.0-beta.8 * prettier * move stuff into multiple files, and stop exporting the extension * lots of doc comments * address agent.ts XXX comments * implement privateVariables simplify API by removing flush() and allowing flush-ers to just call sendReport directly * privateHeaders and error tracking * gzip, signals * fix test * 0.0.0-beta.9 * Error handling for reports * 0.0.0-beta.10 * no need to include boring stacktrace * 0.0.0-beta.11 * tweak error reporting * 0.0.0-beta.12 * package-lock update (npm@6?) * Reduce target report size to 512KB from 4MB. Load testing revealed that protobuf encoding for large FullTraceReports could tie up CPU and reduce p99 request latency (eg, to 200ms from 10ms). Reducing the default target report size spreads out the encoding time and mitigates the impact on latency. If this is not acceptable for all users, we may have to investigate reintroducing agent-side stats aggregation to keep report sizes small. * 0.0.0-beta.13 * Encode Traces as they come in This improves p99 times with little effect on p50 times. It also lets us get rid of the heuristic average trace size estimation. * 0.0.0-beta.14 * support PQ fields * npm audit fix * 0.0.0-beta.15 * ignore coverage * Make the default signature more aggressive We'd rather tell people confused by literal removal to tweak the signature than tell people causing outages to do so. * 0.0.0-beta.16 * Remove obsolete files from graphql-extensions and apollo-engine-reporting * Fix dependencies and configs * Fix apollo-server-cloudflare to import from apollo-server-env * Fix compilation and test configs * Get all tests passing again * Switch to Lerna independent versioning * Polyfill promisify for Node < 8 and load polyfills in tests * ES2016 exponentiation operator is only supported in Node > 6 * add dependency cache for circle * add missing env dependencies in REST datasource
* Initial commit * 0.0.3 * Replace endOffset with duration in resolver calls * 0.0.4 * Fix duration * 0.0.5 * Remove unnecessary schema level resolve function and return schema * Update README * 0.0.6 * Update README * Update dependencies * 0.0.7 * Update README * set package.json to point to this repository (#3) * Update dependencies Fixes #4. * 0.0.8 * Add asynciterable support to tsconfig.json * Skip trace collection when context or context._traceCollector is undefined Fixes #5. * 0.0.9 * Rewrite to use graphql-extensions * 0.0.10 * 0.0.11 * 0.1.0 * Update graphql-extensions dependency and downgrade TS target * 0.1.1 * Update README * Update README * Increase version range for `graphql` peerDependency. (#7) * 0.1.2 * Update dependencies * 0.1.3 * [apollo-bot] Update the Issue/PR Templates with auto label (#9) * Update `graphql` peer dependency range to allow 0.13.x. * dev: Update TypeScript to latest version, v2.7.2. * dev: Update jest & dependencies to latest versions. * dev: Update type definitions for `graphql`, `node` and `jest`. * Allow `undefined` to return from `format`. (#12) * Allow `undefined` to return from `format`. TypeScript 2.7 introduced new "Strict Class Instantiation" rules which, as the name suggests, require properties which are intended to be set (eventually, to a type) be set during construction. Before this change, the `TracingExtension` class was deferred setting these private properties (namely, `startWallTime`, `endWallTime`, `startHrTime` and `duration`), but not during instantiation which required setting marking them as optional and guarding their usage in other methods which might use those (temporarily `undefined`, if even for a tick) properties. For example, the expectation that `format` is _only_ called after `requestDidStart` is not guaranteed with this configuration, even if it is expected under normal operation. Therefore, this change adds the additional guarding and updates the `format` method to return `undefined` in the event that it doesn't have the appropriate data. * Update `graphql-extensions` dependency to `~0.0.9`. Specifically, to take advantage of a type which landed in `graphql-extensions@0.0.9` thanks to https://github.com/apollographql/graphql-extensions/pull/10. * 0.1.4 * Update for graphql-extensions@0.1.0 API (#13) * Upgrade to TypeScript 2.8 * Add tslint and prettier * Update for graphql-extensions@0.1.0 API * 0.2.0-beta.0 * Make work with newest API usage format() now gets called before the requestDidStart() EndHandler. * 0.2.0-beta.1 * remove unused files from tracing package * upgrade packages, fix compilation bugs, and add test
Drop support for Node.js 6 and update TS compilation targets accordingly.
…priately. The `engine` property is just a suggestion, but the `es2017` compilation target should be appropriate for Node.js 8 and higher. This is now appropriate since we've dropped Node.js 6 support in the repository, as was done in #4.
This partially reverts the changes in f74e10eea31a9093168811b8264cdf12bbd72421 which changed from using the native `Object.entries` and `Object.values` methods provided in ES2017 to use `objectEntries` and `objectValues` helpers which mock that functionality for ES2016. As we've now dropped support for Node.js and bumped the compilation target in #4, those abstractions shouldn't be necessary anymore. This preserves the rest of the changes in the aforementioned commit, such as the file moves from `federation` to `composition`.
Drop support for Node.js 6 and update TS compilation targets accordingly.
Drop support for Node.js 6 and update TS compilation targets accordingly.
…priately. The `engine` property is just a suggestion, but the `es2017` compilation target should be appropriate for Node.js 8 and higher. This is now appropriate since we've dropped Node.js 6 support in the repository, as was done in #4.
This partially reverts the changes in f74e10eea31a9093168811b8264cdf12bbd72421 which changed from using the native `Object.entries` and `Object.values` methods provided in ES2017 to use `objectEntries` and `objectValues` helpers which mock that functionality for ES2016. As we've now dropped support for Node.js and bumped the compilation target in #4, those abstractions shouldn't be necessary anymore. This preserves the rest of the changes in the aforementioned commit, such as the file moves from `federation` to `composition`.
With the current focus on small independent microservices in the REST world it would be good to be able to chain multiple apollo servers together.
For instance you may have a single user repository in the company. A apollo server would expose a graphql endpoint to query and mutate these users.
An app server then needs to query this database for user name and address etc. the app server would expose via proxy a subset of queries from the user repository server with the schema being read from their.
The graphql proxy connector could then have extension points to transform the incoming/outcoming queries and results from the chained server.
The text was updated successfully, but these errors were encountered: