Middleware for Wrangler

[cameron-robey]

The current situation
With middleware recently shipping for wrapping workers (#1466), it is worth reviewing how middleware is and should be handled for wrangler dev. Middleware can be shoe-horned into the proxy server currently, but a more thought out and “robust” solution would be worth thinking about as more features need to access this. Additionally, it is worth thinking about where the best place for middleware to take place is (both for speed, and to avoid unnecessary additional computation at the edge).

One of the main reasons for needing middleware at both of these scopes is due to certain resources only being accessible in certain scopes. KV, DO and other storage products are only accessible from the edge, whereas local files, such as source-maps cannot be accessed from the edge. Additionally we may want to prioritise running as much as possible locally to reduce load (albeit potentially very small) at the edge where possible.

Example features that require middleware, and where we should be handling them:

1. Live-reload (#1167)
   There is no requirement for anything to be executed at the edge here. The two main requirements for this feature are script injection and a web socket server, both of which can be inserted completely locally.

2. Scheduled worker testing (#570)
   This needs to take place on the edge, as we have to trigger a scheduled event from a fetch event.

3. Custom error pages (#441)
   Instead of showing a 1xxx error page when a worker errors at runtime, we should show a stacktrace. This may require executing scripts both at the edge (for the error catching, unless we can use the errors sent for the inspector), and locally (to be able to interpret sourcemaps).

4. Serving static assets from KV
   Needs to run at the edge, as it needs to access KV.

A few more use cases can be found at (#1466). Some of the miniflare features to be brought into wrangler will require local middleware too.

Proposal
A “formal” way of handling local middleware for the proxy server within wrangler.

There are two logical positions for local middleware to be placed, both to handle outgoing and incoming events, as shown below.

The more useful middleware at the current time is for the incoming traffic (that has been returned from the edge). This will allow for modifying the response to allow for script injection, or modifying the error response page. Outgoing middleware could be useful for insertion or editing of headers.
Middleware will need to be dynamically used based on the environment and properties of the request/response (for example content-type).

Looking forward - the open source run time
Depending on the exact implementation of the open source runtime, it may be possible to run both local and remote dev through the same proxy server, meaning that middleware can be used “out of the box” with both modes. This would simplify future development as then many features would no longer require development for each (for example scheduled worker testing, or custom error pages).

[IgorMinar]

we may want to prioritise running as much as possible locally to reduce load (albeit potentially very small) at the edge where possible.

I don't believe that this is a concern right now. We do care about efficiency, but the load generated by wrangler dev is insignificant, so I recommend that we prioritize DX over efficiency here.

[IgorMinar]

One of the main reasons for needing middleware at both of these scopes is due to certain resources only being accessible in certain scopes. KV, DO and other storage products are only accessible from the edge, whereas local files, such as source-maps cannot be accessed from the edge.

This is a great paragraph that captures the motivation. I suggest that you highlight it more.

[IgorMinar]

4. Serving static assets from KV
   Needs to run at the edge, as it needs to access KV.

I believe that his is not right. For static assets we want to load and serve the assets from the local filesystem so that we don't need to upload and reupload all assets into KV when the change locally. So we'd actually use the local version of the middleware for this purpose.

[IgorMinar]

currently this functionality is built into the proxy server: https://github.com/cloudflare/wrangler2/blob/c18c60eeacca27656f0e21f1bdcfc0e1298343c3/packages/wrangler/src/proxy.ts#L258

this is a good candidate for an extraction into a local middleware

[IgorMinar]

handle outgoing and incoming events, as shown below.

May I suggest that we simplify the terminology by sticking to request and response terms to describe the stages in the middleware? Or is there a reason to go with "outgoing" and "incoming" terms? I find that they could be interpreted in an inconsistent way.

It might also be useful to not explicitly separate the request and response handlers, so that you can create an middleware that has control over both intercepting the request and response.

[IgorMinar]

Great write up @cameron-robey! Can you please check my understanding of our proposal? This is how I understand it:

In addition to the current Wrangler middleware support that wraps the worker script at build time and is uploaded along with the user code to the edge, you are proposing that we add a concept of local middleware, which is part of the local development server proxy and intercepts all requests and responses locally.

This middleware could use used for adding live-reload functionality to wrangler dev, error stack rewriting using locally available source maps, as well as for static asset serving which is currently directly built into the proxy.

We likely want this to be an internal-only feature for now (not yet exposed to external developers) — just like the current built-time middleware.

Is that right? If so, that sounds great to me.