[Enhancement]: Properly type routes and validate all structures

[jonbarrow]

Checked Existing

• I have checked the repository for duplicate issues.

What enhancement would you like to see?

Currently routes remain untyped and only loose validation is done, mostly on the parts which we actually use. We don't really make many sanity checks for other values.

Additionally we have functions like getValueFromHeaders and getValueFromQueryString as hacky work arounds to the Express type system without actually understanding how it works.

Instead we should be actually typing the routes and validating them using something like Zod.

Any other details to share? (OPTIONAL)

As an example, take the NNAS request for getting your access token:

POST /v1/api/oauth20/access_token/generate HTTP/1.1
Host: account.pretendo.cc
X-Nintendo-Platform-ID: 1
X-Nintendo-Device-Type: 2
X-Nintendo-Device-ID: REDACTED
X-Nintendo-Serial-Number: REDACTED
X-Nintendo-System-Version: 0270
X-Nintendo-Region: 2
X-Nintendo-Country: US
Accept-Language: en
X-Nintendo-Client-ID: a2efa818a34fa16b8afbc8a74eba3eda
X-Nintendo-Client-Secret: c91cdb5658bd4954ade78533a339cf9a
Accept: */*
X-Nintendo-FPD-Version: 0000
X-Nintendo-Environment: L1
X-Nintendo-Title-ID: 0005001010040100
X-Nintendo-Unique-ID: 00401
X-Nintendo-Application-Version: 0115
X-Nintendo-Device-Cert: REDACTED
Content-type: application/x-www-form-urlencoded
Content-Length: 127

grant_type=password&user_id=PN_Jon&password=REDACTED&password_type=hash

This can be typed and validated like so:

import express, { Request, Response, RequestHandler } from 'express';
import { z } from 'zod';
import xmlbuilder from 'xmlbuilder';

// * Custom type to represent our strictly typed requests
type TypedRequest<
	P,
	ResBody,
	ReqBody,
	ReqQuery,
	ReqHeaders
> = Request<P, ResBody, ReqBody, ReqQuery> & {
	headers: ReqHeaders;
};

// * Wrapper function that validates the Zod schemas and enriches the
// * request object to have strict types
function TypedHandler<
	ParamsSchema extends z.ZodTypeAny,
	HeadersSchema extends z.ZodTypeAny,
	BodySchema extends z.ZodTypeAny,
	QuerySchema extends z.ZodTypeAny,
	ResBody = any
>(
	paramsSchema: ParamsSchema,
	headersSchema: HeadersSchema,
	bodySchema: BodySchema,
	querySchema: QuerySchema,
	handler: (
		request: TypedRequest<
			z.infer<ParamsSchema>,
			ResBody,
			z.infer<BodySchema>,
			z.infer<QuerySchema>,
			z.infer<HeadersSchema>
		>,
		response: Response
	) => void
): RequestHandler {
	return (request, response) => {
		try {
			// * Validate all parts of the request
			request.params = paramsSchema.parse(request.params);
			(request as any).headers = headersSchema.parse(request.headers);
			request.body = bodySchema.parse(request.body);
			request.query = querySchema.parse(request.query);

			// * Cast the request and call the real handler
			handler(
				request as TypedRequest<
					z.infer<ParamsSchema>,
					ResBody,
					z.infer<BodySchema>,
					z.infer<QuerySchema>,
					z.infer<HeadersSchema>
				>,
				response
			);
		} catch (error: any) {
			response.status(400).send(xmlbuilder.create({
				errors: {
					error: {
						cause: 'Bad Request',
						code: '1600',
						message: 'Unable to process request'
					}
				}
			}).end());
		}
	};
}

const app = express();
app.use(express.urlencoded({ extended: true }));
app.use(express.json());

const pathParamsSchema = z.object({}); // * No path params for this route

const headersSchema = z.object({
	'x-nintendo-platform-id': z.enum([
		'0', // * 3DS
		'1'  // * Wii U
	]),
	'x-nintendo-device-type': z.literal('2'), // * 1=Debug, 2=Retail
	'x-nintendo-device-id': z.string().regex(/^\d+$/),
	'x-nintendo-serial-number': z.string(),
	'x-nintendo-system-version': z.enum([
		'0320', // * 3DS
		'0270'  // * Wii U
	]),
	'x-nintendo-region': z.enum([
		'1',  // * JPN
		'2',  // * USA
		'4',  // * EUR
		'8',  // * AUS
		'16', // * CHN
		'32', // * KOR
		'64'  // * TWN
	]),
	'x-nintendo-country': z.string(), // * Variable
	'accept-language': z.string(), // * Variable
	'x-nintendo-client-id': z.enum([
		'ea25c66c26b403376b4c5ed94ab9cdea', // * 3DS
		'a2efa818a34fa16b8afbc8a74eba3eda'  // * Wii U
	]),
	'x-nintendo-client-secret': z.enum([
		'd137be62cb6a2b831cad8c013b92fb55', // * 3DS
		'c91cdb5658bd4954ade78533a339cf9a'  // * Wii U
	]),
	'accept': z.literal('*/*'), // * Always this
	'x-nintendo-fpd-version': z.literal('0000'), // * Always this?
	'x-nintendo-environment': z.literal('L1'), // * Retail console
	'x-nintendo-title-id': z.string(), // * Variable
	'x-nintendo-unique-id': z.string(), // * Variable
	'x-nintendo-application-version': z.string(), // * Variable
	'x-nintendo-device-cert': z.string(), // * Variable
	'content-type': z.literal('application/x-www-form-urlencoded'), // * Always this
	'content-length': z.string().regex(/^\d+$/)

});

const bodySchema = z.discriminatedUnion('grant_type', [
	z.object({
		grant_type: z.literal('password'),
		user_id: z.string(),
		password: z.string(),
		password_type: z.literal('hash'),
	}),
	z.object({
		grant_type: z.literal('refresh_token'),
		refresh_token: z.string(),
	}),
]);

const querySchema = z.object({}); // * No query params for this route

app.post('/v1/api/oauth20/access_token/generate', TypedHandler(pathParamsSchema, headersSchema, bodySchema, querySchema, (request, response) => {
	const username = request.body.grant_type === 'password' ? request.body.user_id : 'default';

	response.send(`Token generated for ${username}`);
}));

app.listen(3000, () => console.log('Server running on port 3000'));

Of course things like TypedRequest and TypedHandler would be defined somewhere more global. This gives us intellisense and strict typing on the request data inside the route handler and automatically rejects requests which don't match the Zod schemas.

However this comes with 2 caveats which I'm not sure how best to work around right now:

1. Where do these schemas get defined? Right now our routes are defined in whatever.ts files where whatever comes from /v1/api/WHATEVER. Each file can have MANY routes, and most do have many. If the Zod schemas are defined in these files as well then the files will get very large and unwieldy to work in. So maybe in their own files like we do for types?
2. This removes some flexibility when it comes to more granular error responses. The example I gave only supports sending error 1600 but the server is expected to send back specific errors in some cases. For example if the username is missing from the request body during a password login then the server should send back error 0002, which can't currently happen. TypedHandler is supposed to be generic so we can't directly have it handle specific errors on its own. But if we had another "error handler" function then routes would look something like:

app.post('/v1/api/oauth20/access_token/generate', TypedHandler(pathParamsSchema, headersSchema, bodySchema, querySchema, (request, response) => {
	const username = request.body.grant_type === 'password' ? request.body.user_id : 'default';

	response.send(`Token generated for ${username}`);
}), () => {
	// * Do error stuff here
});

which might not be nice to work with? Idk, need options.

I wonder if it would be worth making this TypedHandler system it's own library as well, so we can use it in our other servers?