Experimental support for incremental delivery (@defer/@stream)

Requires a pre-release of graphql@17.

More detailed commit message to come.

Fixes #6671.

.changeset/chilled-cows-drum.md

@@ -0,0 +1,7 @@
+---
+'@apollo/server-integration-testsuite': patch
+'@apollo/server-plugin-response-cache': patch
+'@apollo/server': patch
+---
+
+Experimental support for incremental delivery (`@defer`/`@stream`) when combined with a prerelease of `graphql-js`.

.changeset/thirty-donkeys-sing.md

@@ -0,0 +1,7 @@
+---
+'@apollo/server-integration-testsuite': patch
+'@apollo/server-plugin-response-cache': patch
+'@apollo/server': patch
+---
+
+Support application/graphql-response+json content-type if requested via Accept header, as per graphql-over-http spec.

.circleci/config.yml

@@ -69,6 +69,22 @@ jobs:
  - setup-node
  - run: npm run test:smoke
 
+ Full incremental delivery tests with graphql-js 17 canary:
+ docker:
+ - image: cimg/base:stable
+ environment:
+ INCREMENTAL_DELIVERY_TESTS_ENABLED: t
+ steps:
+ - setup-node:
+ node-version: "18"
+ # Install a prerelease of graphql-js 17 with incremental delivery support.
+ # --legacy-peer-deps because nothing expects v17 yet.
+ # --no-engine-strict because Node v18 doesn't match the engines fields
+ # on some old stuff.
+ - run: npm i --legacy-peer-deps --no-engine-strict graphql@17.0.0-alpha.1.canary.pr.3361.04ab27334641e170ce0e05bc927b972991953882
+ - run: npm run test:ci
+ - run: npm run test:smoke
+
  Prettier:
  docker:
  - image: cimg/base:stable
@@ -143,4 +159,5 @@ workflows:
  - ESLint
  - Spell check
  - Smoke test built package
+ - Full incremental delivery tests with graphql-js 17 canary
  - Changesets

docs/source/api/apollo-server.mdx

@@ -589,7 +589,7 @@ In some circumstances, Apollo Server calls `stop` automatically when the process
 The async `executeOperation` method is used primarily for [testing GraphQL operations](../testing/testing/#testing-using-executeoperation) through Apollo Server's request pipeline _without_ sending an HTTP request.
 
 ```js
-const result = await server.executeOperation({
+const response = await server.executeOperation({
  query: 'query SayHelloWorld($name: String) { hello(name: $name) }',
  variables: { name: 'world' },
 });
@@ -601,6 +601,12 @@ The `executeOperation` method takes two arguments:
  - Supported fields are listed in the table below.
 - The second optional argument is used as the operation's [context value](../data/resolvers/#the-context-argument). Note, this argument is only optional if your server _doesn't_ expect a context value (i.e., your server uses the default context because you didn't explicitly provide another one).
 
+The `response` object returned from `executeOperation` is a `GraphQLResponse`, which has `body` and `http` fields.
+
+Apollo Server 4 supports incremental delivery directives such as `@defer` and `@stream` (when combined with an appropriate version of `graphql-js`), and so the structure of `response.body` can represent either a single result or multiple results. `response.body.kind` is either `'single'` or `'incremental'`. If it is `'single'`, then incremental delivery has not been used, and `response.body.singleResult` is an object with `data`, `errors`, and `extensions` fields. If it is `'incremental'`, then `response.body.initialResult` is the initial result of the operation, and `response.body.subsequentResults` is an async iterator that will yield subsequent results. (The precise structure of `initialResult` and `subsequentResults` is defined by `graphql-js` and may change between the current pre-release of `graphql-js` v17 and its final release; if you write code that processes these values before `graphql-js` v17 has been released you may have to adapt it when the API is finalized.)
+
+The `http` field contains an optional numeric `status` code and a `headers` map specifying any HTTP status code and headers that should be set.
+
 Below are the available fields for the first argument of `executeOperation`:
 
 ### Fields

docs/source/data/errors.mdx

@@ -660,7 +660,8 @@ const setHttpPlugin = {
  return {
  async willSendResponse({ response }) {
  response.http.headers.set('custom-header', 'hello');
- if (response?.result?.errors?.[0]?.extensions?.code === 'TEAPOT') {
+ if (response.body.kind === 'single' &&
+ response.body.singleResult.errors?.[0]?.extensions?.code === 'TEAPOT') {
  response.http.status = 418;
  }
  },

docs/source/integrations/building-integrations.md

@@ -20,7 +20,7 @@ requests and responses between a web framework's native format to the format use
 
 > For more examples, see these Apollo Server 4 [integrations demos for Fastify and Lambda](https://github.com/apollographql/server-v4-integration-demos/tree/main/packages).
 
-If you are building a serverless integration, we **strongly recommend** prepending your function name with the word `start` (e.g., `startServerAndCreateLambdaHandler(server)`). This naming convention helps maintain Apollo Server's standard that every server uses a function or method whose name contains the word `start` (such as `startStandaloneServer(server)`. 
+If you are building a serverless integration, we **strongly recommend** prepending your function name with the word `start` (e.g., `startServerAndCreateLambdaHandler(server)`). This naming convention helps maintain Apollo Server's standard that every server uses a function or method whose name contains the word `start` (such as `startStandaloneServer(server)`.
 
 ### Main function signature
 
@@ -225,30 +225,37 @@ interface HTTPGraphQLHead {
  headers: Map<string, string>;
 }
 
-type HTTPGraphQLResponse = HTTPGraphQLHead &
- (
- | {
- completeBody: string;
- bodyChunks: null;
- }
- | {
- completeBody: null;
- bodyChunks: AsyncIterableIterator<HTTPGraphQLResponseChunk>;
- }
- );
+type HTTPGraphQLResponseBody =
+ | { kind: 'complete'; string: string }
+ | { kind: 'chunked'; asyncIterator: AsyncIterableIterator<string> };
+
+
+type HTTPGraphQLResponse = HTTPGraphQLHead & {
+ body: HTTPGraphQLResponseBody;
+};
 ```
 
-The Express implementation uses the `res` object to update the response
-with the appropriate status code and headers, and finally sends the body:
+Note that a body can either be "complete" (a complete response that can be sent immediately with a `content-length` header), or "chunked", in which case the integration should read from the async iterator and send each chunk one at a time. (This typically will use `transfer-encoding: chunked`, though your web framework may handle that for you automatically.) If your web environment does not support streaming responses (as in some serverless function environments like AWS Lambda), you can return an error response if a chunked body is received.
+
+The Express implementation uses the `res` object to update the response with the appropriate status code and headers, and finally sends the body. Note that in Express, `res.send` will send a complete body (including calculating the `content-length` header), and `res.write` will use `transfer-encoding: chunked`.
 
 ```ts
 for (const [key, value] of httpGraphQLResponse.headers) {
  res.setHeader(key, value);
 }
 res.statusCode = httpGraphQLResponse.status || 200;
-res.send(httpGraphQLResponse.completeBody);
+
+if (httpGraphQLResponse.body.kind === 'complete') {
+ res.send(httpGraphQLResponse.body.string);
+ return;
+}
+
+for await (const chunk of httpGraphQLResponse.body.asyncIterator) {
+ res.write(chunk);
+}
+res.end();
 ```
 
 ## Additional resources
 
-The [`@apollo/server-integration-testsuite`](https://www.npmjs.com/package/@apollo/server-integration-testsuite) provides a set of Jest tests for authors looking to test their Apollo Server integrations. 
+The [`@apollo/server-integration-testsuite`](https://www.npmjs.com/package/@apollo/server-integration-testsuite) provides a set of Jest tests for authors looking to test their Apollo Server integrations.

docs/source/migration.mdx

@@ -298,7 +298,7 @@ In Apollo Server 3, the `apollo-server-core` package exports built-in plugins, l
 
 In Apollo Server 4, these built-in plugins are part of the main `@apollo/server` package, which also imports the `ApolloServer` class. The `@apollo/server` package exports these built-in plugins with deep exports. This means you use deep imports for each built-in plugin, enabling you to evaluate only the plugin you use in your app and making it easier for bundlers to eliminate unused code.
 
-There's one exception: the `ApolloServerPluginLandingPageGraphQLPlayground` plugin is now in its own package `@apollo/server-plugin-landing-page-graphql-playground`, which you can install separately. 
+There's one exception: the `ApolloServerPluginLandingPageGraphQLPlayground` plugin is now in its own package `@apollo/server-plugin-landing-page-graphql-playground`, which you can install separately.
 
 This plugin installs the [unmaintained](https://github.com/graphql/graphql-playground/issues/1143) GraphQL Playground project as a landing page and is provided for compatibility with Apollo Server 2. This package will **not** be supported after Apollo Server 4 is released. We strongly recommend you switch to Apollo Server's 4's [default landing page](./api/plugin/landing-pages), which installs the actively maintained Apollo Sandbox.
 
@@ -727,11 +727,12 @@ new ApolloServer<MyContext>({
  return {
  async willSendResponse(requestContext) {
  const { response } = requestContext;
- // Augment response with an extension, as long
- // as the operation actually executed.
- if ('data' in response.result) {
- response.result.extensions = {
- ...(response.result.extensions),
+ // Augment response with an extension, as long as the operation
+ // actually executed. (The `kind` check allows you to handle
+ // incremental delivery responses specially.)
+ if (response.body.kind === 'single && 'data' in response.body.singleResult) {
+ response.body.singleResult.extensions = {
+ ...(response.body.singleResult.extensions),
  hello: 'world',
  };
  }
@@ -1112,6 +1113,8 @@ In Apollo Server 3, you can indirectly specify an operation's context value by p
 
 In Apollo Server 4, the `executeOperation` method optionally receives a context value directly, bypassing your `context` function. If you want to test the behavior of your `context` function, we recommend running actual HTTP requests against your server.
 
+Additionally, the [structure of the returned `GraphQLResponse` has changed](#graphqlresponse), as described below.
+
 So a test for Apollo Server 3 that looks like this:
 
 <MultiCodeBlock>
@@ -1127,7 +1130,7 @@ const server = new ApolloServer({
  context: async ({ req }) => ({ name: req.headers.name }),
 });
 
-const { result } = await server.executeOperation({
+const result = await server.executeOperation({
  query: 'query helloContext { hello }',
 }, {
  // A half-hearted attempt at making something vaguely like an express.Request,
@@ -1158,13 +1161,17 @@ const server = new ApolloServer<MyContext>({
  },
 });
 
-const { result } = await server.executeOperation({
+const { body } = await server.executeOperation({
  query: 'query helloContext { hello }',
 }, {
  name: 'world',
 });
 
-expect(result.data?.hello).toBe('Hello world!'); // -> true
+// Note the use of Node's assert rather than Jest's expect; if using
+// TypeScript, `assert`` will appropriately narrow the type of `body`
+// and `expect` will not.
+assert(body.kind === 'single');
+expect(body.singleResult.data?.hello).toBe('Hello world!'); // -> true
 ```
 
 </MultiCodeBlock>
@@ -1425,23 +1432,16 @@ Specifically, the `http` field is now an `HTTPGraphQLRequest` type instead of a
 
 Apollo Server 4 refactors the [`GraphQLResponse` object](https://github.com/apollographql/apollo-server/blob/version-4/packages/server/src/externalTypes/graphql.ts#L25), which is available to plugins as `requestContext.response` and is the type `server.executeOperation` returns.
 
-The `data`, `errors`, and `extensions` fields are now nested within an object returned by the `result` field:
+In Apollo Server 3, the `data`, `errors`, and `extensions` fields existed at the top level, right beside `http`.
 
-```ts disableCopy
-export interface GraphQLResponse {
- // The below result field contains an object with the
- // data, errors, and extensions fields
- result: FormattedExecutionResult;
- http: HTTPGraphQLHead;
-}
-```
+Because Apollo Server 4 supports incremental delivery directives such as `@defer` and `@stream` (when combined with an appropriate version of `graphql-js`), the structure of the response can now represent either a single result or multiple results, so these fields no longer exist at the top level of `GraphQLResponse`.
+
+Instead, there is a `body` field at the top level of `GraphQLResponse`. `response.body.kind` is either `'single'` or `'incremental'`. If it is `'single'`, then incremental delivery has not been used, and `response.body.singleResult` is an object with `data`, `errors`, and `extensions` fields. If it is `'incremental'`, then `response.body.initialResult` is the initial result of the operation, and `response.body.subsequentResults` is an async iterator that will yield subsequent results. (The precise structure of `initialResult` and `subsequentResults` is defined by `graphql-js` and may change between the current pre-release of `graphql-js` v17 and its final release; if you write code that processes these values before `graphql-js` v17 has been released you may have to adapt it when the API is finalized.)
 
 Additionally, the `data` and `extensions` fields are both type `Record<string, unknown>`, rather than `Record<string, any>`.
 
 The value of `http.headers` is now a `Map` rather than a Fetch API `Headers` object. All keys in this map must be lower-case; if you insert any header name with capital letters, it will throw.
 
-> We plan to implement experimental support for incremental delivery (`@defer`/`@stream`) before the v4.0.0 release and expect this to change the structure of `GraphQLResponse` further.
-
 ### `plugins` constructor argument does not take factory functions
 
 In Apollo Server 3, each element of the `plugins` array provided to `new ApolloServer` could either be an `ApolloServerPlugin` (ie, an object with fields like `requestDidStart`) or a zero-argument "factory" function returning an `ApolloServerPlugin`.
@@ -1525,7 +1525,7 @@ Apollo Server supports [batching HTTP requests](./workflow/requests/#batching),
 
 In Apollo Server 4, you must explicitly enable this feature by passing `allowBatchedHttpRequests: true` to the `ApolloServer` constructor.
 
-Not all GraphQL clients support HTTP batching, and batched requests will not support incremental delivery when Apollo Server implements that feature. HTTP batching can help performance by sharing a `context` object across operations, but it can make it harder to understand the amount of work any given request does.
+Not all GraphQL clients support HTTP batching, and batched requests do not support incremental delivery. HTTP batching can help performance by sharing a `context` object across operations, but it can make it harder to understand the amount of work any given request does.
 
 
 ### Default cache is bounded

docs/source/testing/testing.mdx

@@ -50,14 +50,18 @@ it('returns hello with the provided name', async () => {
  variables: { name: 'world' },
  });
 
- expect(response.result.errors).toBeUndefined();
- expect(response.result.data?.hello).toBe('Hello world!');
+ // Note the use of Node's assert rather than Jest's expect; if using
+ // TypeScript, `assert`` will appropriately narrow the type of `body`
+ // and `expect` will not.
+ assert(body.kind === 'single');
+ expect(response.body.singleResult.errors).toBeUndefined();
+ expect(response.body.singleResult.data?.hello).toBe('Hello world!');
 });
 ```
 
 </MultiCodeBlock>
 
-Note that when testing, any errors in parsing, validating, and executing your GraphQL operation are returned in the `errors` field of the operation result. As with any GraphQL response, these errors are not _thrown_.
+Note that when testing, any errors in parsing, validating, and executing your GraphQL operation are returned in the nested `errors` field of the result. As with any GraphQL response, these errors are not _thrown_.
 
 You don't need to start your server before calling `executeOperation`. The server instance will start automatically and throw any startup errors.
 

docs/source/workflow/requests.md

@@ -86,3 +86,9 @@ Unlike with `POST` requests, `GET` requests do not require a `Content-Type` head
 - A non-empty `Apollo-Require-Preflight` header
 
 For more details, see [the CSRF prevention documentation](../security/cors#preventing-cross-site-request-forgery-csrf).
+
+## Incremental delivery (experimental)
+
+Incremental delivery is a [Stage 2: Draft Proposal](https://github.com/graphql/graphql-spec/pull/742) to the GraphQL specification which adds `@defer` and `@stream` executable directives. These directives allow clients to specify that parts of an operation can be sent after an initial response, so that slower fields do not delay all other fields. As of September 2022, the `graphql` library (also known as `graphql-js`) upon which Apollo Server is built implements incremental delivery only in the unreleased major version 17. If a pre-release of `graphql@17` is installed in your server, Apollo Server 4 can execute these incremental delivery directives and provide streaming [`multipart/mixed`](https://github.com/graphql/graphql-over-http/blob/main/rfcs/IncrementalDelivery.md) responses.
+
+Clients sending operations with incremental delivery directives need to explicitly indicate that they are expecting to receive `multipart/mixed` responses in an `accept` header. Moreover, because incremental delivery has not yet been finalized in the GraphQL spec and may change before the final version, they need to specify that they expect the particular response format that Apollo Server produces today via a `deferSpec` parameter. Specifically, clients prepared to accept incremental delivery responses should send an `accept` header like `multipart/mixed; deferSpec=20220824`. Note that this header implies that *only* multipart responses should be accepted; typically, clients will send an accept header like `multipart/mixed; deferSpec=20220824, application/json` indicating that either multipart or single-part responses are acceptable.