Sharing common error types between services

[lawrencejones]

Hey all!

We're about to begin using Goa as the foundation for our API. One of the first things we wanted to get sorted is errors, but we're having some trouble with this.

We want all errors to use the same (custom) structure, which would look like this:

{
  "error": {
    "type": "authentication_error",
    "status": 401,
    "request_id": "23BF0031F7B0_0A1A05711F92_6116AE3E_5AF30001",
    "errors": [
      {
        "code": "missing_authorization_header",
        "message": "The 'Authorization' header needs to be set and contain a valid API token",
      },
    ],
  }
}

We can define this as a Type("ErrorResponse") in the API design, and each error type would map to a specific HTTP status code.

As errors will be consistent across services, we'd have each service defined like so (probably via a helper method, like CommonService):

var _ = Service("Roles", func() {
	Error("invalid_request_error", ErrorResponse)
	Error("authentication_error", ErrorResponse)
	Error("not_found", ErrorResponse)
	Error("validation_error", ErrorResponse)
	Error("api_error", ErrorResponse)

	HTTP(func() {
		Response("invalid_request_error", StatusBadRequest)     // 400
		Response("authentication_error", StatusUnauthorized)    // 401
		Response("not_found", StatusNotFound)                   // 404
		Response("validation_error", StatusUnprocessableEntity) // 422
		Response("api_error", StatusInternalServerError)        // 500
	})
})

This causes all services to generate ErrorResponse structs: for the Roles service defined above, you'd expect api/gen/roles/service.go to contain the ErrorResponse definition.

So what's the problem?

We want all service implementations to borrow from the same set of error constructors, allowing us to more succinctly generate errors in our handlers.

An ideal situation would be:

// Imagine we have an Errors service, defined for the sake of generating
// service-independent error types.
var (
	ErrorNotFound = &errors.ErrorResponse{
		Status: http.StatusNotFound,
		Type:   "not_found",
		Errors: []*errors.Error{
			{
				Code:    "resource_not_found",
				Message: "Resource not found",
			},
		},
	}
)

// Implementing a Show method on the Roles service
func (svc *rolesService) Show(ctx context.Context, payload *roles.ShowPayload) (*roles.Role, error) {
	return nil, NotFound // if we can't find the resource
}

Sadly this will fail, as the Roles service has created a error encoder that asserts on the roles.ErrorResponse type, which won't match the type we've generated in the errors package.

Why can't you just use roles.ErrorResponse?

These errors are fairly verbose, and we want to build a library of helper functions to construct the more complex ones.

We don't want to copy-and-paste these helpers for each of our (many) services, and we want to create application middlewares that are generic across services- such as a generic resource finder, which would want to return a common NotFound error.

So what are you asking?

How can we share the same error types across services? We definitely want a common error type so we can create sensible builders, but are we going about this the wrong way?

Might we want to bypass the default error encodings entirely?

Thanks for your help!

Lawrence

[raphael]

Hello!

It is possible to use custom error types as described in Handling Errors. Generated Goa error types can also be customized as described in Error Handling (might make sense to merge these two pages into one!).

The following design would make it possible for the user code to return error responses corresponding to the data structure described above:

package design

import . "goa.design/goa/v3/dsl"

var _ = Service("roles", func() {
	Error("invalid_request_error", ErrorResponse) // Sets the error type to the ErrorResponse data structure
	Error("authentication_error", ErrorResponse)
	Error("not_found", ErrorResponse)
	Error("validation_error", ErrorResponse)
	Error("api_error", ErrorResponse)

	HTTP(func() {
		Response("invalid_request_error", StatusBadRequest)     // 400
		Response("authentication_error", StatusUnauthorized)    // 401
		Response("not_found", StatusNotFound)                   // 404
		Response("validation_error", StatusUnprocessableEntity) // 422
		Response("api_error", StatusInternalServerError)        // 500
	})

	Method("method", func() {
		Payload(String)
		HTTP(func() {
			GET("/")
		})
	})
})

var ErrorResponse = ResultType("error_response", func() {
	Attributes(func() {
		Attribute("type", String, "Type of error", func() {
			Meta("struct:error:name")
			Example("authentication_error")
		})
		Attribute("status", Int, "HTTP status code", func() {
			Example(401)
		})
		Attribute("request_id", String, "Request ID", func() {
			Example("123456789")
		})
		Attribute("errors", ArrayOf(ErrorDetails), "List of errors")
		Required("type", "status", "request_id", "errors")
	})
})

var ErrorDetails = Type("error", func() {
	Attribute("code", String, "Error code", func() {
		Example("missing_authorization_header")
	})
	Attribute("message", String, "Error message", func() {
		Example("The 'Authorization' header needs to be set and contain a valid API token")
	})
	Required("code", "message")
})

The user code can then return these errors by instantiating the generated ErrorResponse struct:

func (s *rolessvc) Method(ctx context.Context, p string) error {
	// Convoluted example...
	if p != "secret" {
		err := &roles.Error{Code: "invalid_secret", Message: "Provided secret is invalid"}
		return &roles.Error{Type: "unauthorized_request", Status: 401, RequestID: "123", Errors: []&roles.Error{err}}
	}
	return nil
}

There is still a need for overriding the format of Goa generated errors. This can be accomplished by providing a custom error formatter as shown below:

func customErrorResponse(err error) Statuser {
	if serr, ok := err.(*goa.ServiceError); ok {
		switch serr.Name {
			case "missing_field":
			return missingFieldError(serr.Message)
		default:
			// Use Goa default
			return goahttp.NewErrorResponse(err)
		}
	}
	// Use Goa default for all other error types
	return goahttp.NewErrorResponse(err)
}

Where missingFieldError can use the ServiceError data structure to build a ErrorResponse. This could be used to override all the possible errors listed in error.go.

The custom error formatter is set when creating the HTTP server or handler in the main package:

rolesServer = rolessvr.New(rolesEndpoints, mux, dec, enc, eh, customErrorResponse)

Looking at #2902 I'm guessing that you are already using a custom error formatter and that this PR is needed to implement it?