How to setup error monitoring (Sentry / Bugsnag etc)

[asgeo1]

It doesn't seem possible with Keystone 6, to properly setup error monitoring, i.e. with Sentry or Bugsnag etc.

extendExpressApp is just not flexible enough, because it only allows adding middleware to the Express app, before the Keystone/Apollo middleware is added, and not after.

This is what I have currently based off the Sentry express documentation

import * as Sentry from '@sentry/node';
import { ProfilingIntegration } from '@sentry/profiling-node';

export default config({
  ...
  server: {
    ...
    extendExpressApp: (app, commonContext) => {
      Sentry.init({
        dsn: process.env.SENTRY_DSN,
        environment: process.env.SENTRY_ENVIRONMENT,
        integrations: [
          // enable HTTP calls tracing
          new Sentry.Integrations.Http({ tracing: true }),
          // enable Express.js middleware tracing
          new Sentry.Integrations.Express({ app }),
          new ProfilingIntegration()
        ],
        // Performance Monitoring
        tracesSampleRate: 1.0,
        // Set sampling rate for profiling - this is relative to tracesSampleRate
        profilesSampleRate: 1.0
      });

      // The request handler must be the first middleware on the app
      app.use(Sentry.Handlers.requestHandler());

      // TracingHandler creates a trace for every incoming request
      app.use(Sentry.Handlers.tracingHandler());


      ...


      // The error handler must be registered before any other error middleware and after all controllers
      app.use(Sentry.Handlers.errorHandler());

      // Optional fallthrough error handler
      app.use(function onError(_err, _req, res, _next) {
        // The error id is attached to `res.sentry` to be returned
        // and optionally displayed to the user for support.
        res.statusCode = 500;
        res.end(res.sentry + '\n');
      });
    }
  }
})

But I don't think it's correct, as some of that middleware is being added at the wrong point, and should be added after the Apollo routes etc are created inside of createExpressServer

It would be better if there were multiple hooks that could be called, to adjust the Express middleware both before and after Keystone has added it's own middleware.

With the Admin-UI, it also should have it's own error monitoring, separate to the Express backend. I would like to add Sentry/Bugsnag into the Admin UI's Next.js application too, but I can't see any way in which to do that either.

[marekryb]

Hi,

If you want to setup detailed monitoring on graphQL layer, then aside from express monitoring that you already have, I would suggest using apollo-server plugin system. You can add your plugins through graphql.apolloConfig.plugins in your keystone config.

While googling, I found this repo which implements plugin for Sentry. Note though that this is for apollo-server v3 and keystone is now using apollo-server v4. Probably there is some breaking change in plugin interface, but it should be easy enough to adapt.

[asgeo1]

Hey Marek, thanks for the response.

I didn't realise I could capture exceptions from Apollo server directly. Hmm, yes I think that would cover most kinds of Keystone errors, in conjunction with the my original Express monitoring.

I looked into that plugin, and yes, it doesn't work with Apollo v4, and has some other issues too (repeating Sentry.init setup etc).

I'll post a working example below that I'm using.

cheers

[asgeo1]

This appears to work for me

// sentryPlugin.ts

// Based on https://github.com/pointblankdev/apollo-plugin-sentry/blob/master/src/index.ts

import * as Sentry from '@sentry/node';
import { ApolloServerPlugin } from '@apollo/server';
import type { Context } from '.keystone/types';

const sentryPlugin = (name?: string): ApolloServerPlugin<Context> => {
  return {
    // request lifecycle events
    async requestDidStart() {
      /* Within this returned object, define functions that respond
           to request-specific lifecycle events. */
      return {
        async didEncounterErrors(ctx) {
          // If we couldn't parse the operation (usually invalid queries, not necessary?)
          if (!ctx.operation && ctx.errors) {
            for (const err of ctx.errors) {
              Sentry.withScope(scope => {
                name && scope.setTag('service_name', name);
                scope.setExtra('query', ctx.request.query);
                Sentry.captureException(err);
              });
            }
            return;
          }

          if (ctx.errors) {
            for (const err of ctx.errors) {
              // Add scoped report details and send to Sentry
              Sentry.withScope(scope => {
                name && scope.setTag('service_name', name);
                // Annotate whether failing operation was query/mutation/subscription
                if (ctx.operation) {
                  scope.setTag('kind', ctx.operation.operation);
                }

                // Log query and variables as extras (make sure to strip out sensitive data!)
                scope.setExtra('query', ctx.request.query);
                scope.setExtra('variables', ctx.request.variables);

                if (err.path) {
                  // We can also add the path as breadcrumb
                  scope.addBreadcrumb({
                    category: 'query-path',
                    message: err.path.join(' > '),
                    level: 'debug'
                  });
                }

                const transactionId = ctx.request?.http?.headers.get('x-transaction-id');
                if (transactionId) {
                  scope.setTransactionName(transactionId);
                }

                // conditional capture for only prod?
                Sentry.captureException(err);
              });
            }
          }
        }
      };
    }
  };
};

export { sentryPlugin };
export default { sentryPlugin };

// keystone.ts

import * as Sentry from '@sentry/node';
import { ProfilingIntegration } from '@sentry/profiling-node';
import { sentryPlugin } from './sentryPlugin';

export default config({
  ...
  server: {
    ...
    graphql: {
      apolloConfig: {
        plugins: [sentryPlugin()]
      }
    },
    extendExpressApp: (app, commonContext) => {
      Sentry.init({
        dsn: process.env.SENTRY_DSN,
        environment: process.env.SENTRY_ENVIRONMENT,
        integrations: [
          // enable HTTP calls tracing
          new Sentry.Integrations.Http({ tracing: true }),
          // enable Express.js middleware tracing
          new Sentry.Integrations.Express({ app }),
          new ProfilingIntegration()
        ],
        // Performance Monitoring
        tracesSampleRate: 1.0,
        // Set sampling rate for profiling - this is relative to tracesSampleRate
        profilesSampleRate: 1.0
      });

      // The request handler must be the first middleware on the app
      app.use(Sentry.Handlers.requestHandler());

      // TracingHandler creates a trace for every incoming request
      app.use(Sentry.Handlers.tracingHandler());


      ...


      // The error handler must be registered before any other error middleware and after all controllers
      app.use(Sentry.Handlers.errorHandler());

      // Optional fallthrough error handler
      app.use(function onError(_err, _req, res, _next) {
        // The error id is attached to `res.sentry` to be returned
        // and optionally displayed to the user for support.
        res.statusCode = 500;
        res.end(res.sentry + '\n');
      });
    }
  }
})