Fetch requests don't expose Set-Cookie response header

[ottomated]

The following code behaves differently on miniflare VS on a deployed worker:

addEventListener('fetch', event => {
  event.respondWith(handleRequest(event.request))
})
/**
 * Respond with hello worker text
 * @param {Request} request
 */
async function handleRequest(request) {
  const url = new URL(request.url);

  if (url.pathname === '/cookies') {
    return new Response(null, {
      headers: {
        'Set-Cookie': 'foo=bar'
      }
    })
  } else {
    const res = await fetch(`${url.origin}/cookies`);
    return new Response(JSON.stringify(res.headers.get('set-cookie')))
  }
}

• Visiting the /cookies URL returns a set-cookie header with the value foo=bar. Visiting any other URL does a sub-fetch to the /cookies path and returns the value of the set-cookie header in the response body.

In miniflare, this returns null.. In a worker, this behaves as expected - the set-cookie header is exposed to the user.

The source of this is that undici follows the fetch spec closer than workers. The conflict source is undici/lib/fetch/response.js:370, the filterResponse function, which removes forbidden headers from the response such as set-cookie.

[joona]

I'm hitting the same problem while trying to integrate miniflare to our development process.

In our case, this happens when proxying API requests on path /api to our API Origin behind CF Access (Service Token authentication) without doing any modifications to the response. Request is cloned to be able to attach additional headers for service token authentication. However, the problem also happens when doing no header modifications at all and using IP whitelisting on CF Access side.

The same code works perfectly on Worker runtime.

Invocation on both environments is same-site with valid HTTPS and withCredentials on the client side, etc.

While doing my own digging, I also came to conclusion, that those headers are dropped due undici fetch implementation, but didn't find any way to work around this.

[hansede]

I also came to the same conclusion today. Unfortunately this is the last thing preventing me from deploying Miniflare as part of our development environment. When the set-cookie header fails to propagate, Django's csrf_token can't be set, so Django auth doesn't work. This is notably different behavior from Cloudflare Workers.

[hansede]

I did some additional investigation and opened an issue with Undici:
nodejs/undici#1262

[mrbbot]

Hey everyone! 👋 Thanks for raising this and apologies for the delayed response. This is definitely a bug, might be possible to extract this header from internal undici properties like we're doing here:

miniflare/packages/core/src/standards/http.ts

Line 801 in b887877

const internalResponse = baseRes[fetchSymbols.kState].internalResponse;

...although probably worth waiting on nodejs/undici#1262 for a proper fix. 👍

[joona]

Fix for the problem has been now released on undici v5.4.0 release.

nodejs/undici@v5.3.0...v5.4.0

[mrbbot]

Nice! 🎉 I think this was actually fixed in Miniflare 2.5.0 because of this hack:

miniflare/packages/core/src/standards/http.ts

Lines 63 to 68 in 75abe07

	const constants: {
	readonly forbiddenHeaderNames: string[];
	readonly forbiddenResponseHeaderNames: string[];
	} = require("undici/lib/fetch/constants.js");
	constants.forbiddenHeaderNames.length = 0;
	constants.forbiddenResponseHeaderNames.length = 0;

...but I forgot to close this issue. Anyways, upgrading toundici@5.4.0 is the correct fix.

[joona]

Thank you @mrbbot! You beat me to it, I was just about to send PR.

I can also confirm that the hack worked and cookies are passed properly on v2.5.0 as well 👍