Add option to disable cookie persistence in Client/AsyncClient

[fastily]

Initially raised as discussion #1533

I've been using httpx's AsyncClient as a web spider, however I have noticed that cookies are automatically persisted and there's no easy way to disable this. My workaround has been to subclass python's http.cookiejar.CookieJar like so:

from http.cookiejar import CookieJar

class NullCookieJar(CookieJar):
    """A CookieJar that rejects all cookies."""

    def extract_cookies(self, *_):
        """For extracting and saving cookies.  This implementation does nothing"""
        pass

    def set_cookie(self, _):
        """Normally for setting a cookie.  This implementation does nothing"""
        pass

Would it be possible to:

1. add an option to disable cookie persistence to Client/AsyncClient or
2. include this implementation of NullCookieJar in httpx as a utility class?

Thanks!

[karpetrosyan]

Hi!
Perhaps you could remove the Set-Cookie header before it reaches the client instance?

Example:

import httpx


class DisableCookieTransport(httpx.BaseTransport):
    def __init__(self, transport: httpx.BaseTransport):
        self.transport = transport

    def handle_request(self, request: httpx.Request) -> httpx.Response:
        response = self.transport.handle_request(request)
        del response.headers["set-cookie"]
        return response


client = httpx.Client(transport=DisableCookieTransport(httpx.HTTPTransport()))

response = client.get("https://httpbin.org/cookies/set?foo=bar")

[idrissbellil]

For AsyncClient I am guessing the workaround is:

class AsyncDisableCookieTransport(httpx.AsyncBaseTransport):
    def __init__(self, transport: httpx.AsyncBaseTransport):
        self.transport = transport

    async def handle_async_request(self, request: httpx.Request) -> httpx.Response:
        response = await self.transport.handle_async_request(request)
        response.headers.pop("set-cookie", None)
        return response

client = httpx.AsyncClient(transport=AsyncDisableCookieTransport(httpx.AsyncHTTPTransport()))

[tomchristie]

Perhaps this is a common enough use-case that we should add cookies = httpx.NoCookies(), in line with @fastily's suggestion?

[sobolevn]

I would love to see something like #3065 merged. I consider merging cookies by default to be a possible attack vector, so I would love to turn this off, when not needed.

[jschlyter]

+1 for #3065